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Executive  Summary 

The  Reduced  Vertical  Separation  Minima  (RVSM)  simulation  resulted  from  the  North  Atlantic 
Systems  Planning  Group  (NATSPG)  conclusion  to  carry  out  studies  aimed  at  achieving  early 
implementation  of  RVSM  in  the  North  Atlantic  (NAT)  Region.  RVSM  is  an  approved 
International  Civil  Aviation  Organization  (ICAO)  concept  to  reduce  aircraft  vertical  separation 
from  the  Conventional  Vertical  Separation  Minima  (CVSM)  of  2000  ft  to  1000  ft,  between  flight 
levels  (FLs)  290-410,  within  a  designated  portion  of  the  NAT  Region.  In  the  United  States, 
RVSM  studies  are  being  conducted  by  the  Federal  Aviation  Administration  William  J.  Hughes 
Technical  Center  National  Simulation  Capability  RVSM  Experimentation  Working  Group. 

RVSM  Phase  HI  simulation  studies  were  conducted  in  October  1995  at  the  Miami  Air  Route 
Traffic  Control  Center  (ZMA)  Dynamic  Simulation  (DYSIM)  Laboratory.  The  study 
investigated  workload  effects  and  the  feasibility  of  transitioning  aircraft  to  and  from  CVSM  and 
from  and  to  RVSM  altitudes  within  radar  sectors  Rl,  R60,  R62,  and  R63  under  various  traffic 
conditions.  Experimental  findings  and  conclusions  from  the  simulations  were  provided  to  Air 
Traffic  organizations  to  assist  in  defining  geographical  areas  and  procedures  for  RVSM 
transitioning,  thereby  enhancing  the  flexibility  and  efficiency  of  the  National  Airspace  System. 

Phase  in  simulations  investigated  1 1  different  traffic  scenarios.  Scenarios  varied  by  sector 
configuration  and  separation  minima.  Air  Traffic  Control  specialists  and  DYSIM  training 
specialists  developed  traffic  scenarios  based  on  actual  recorded  traffic  flows.  Currently  certified 
and  active  ZMA  controllers  staffed  the  radar,  the  non-radar,  and  the  assistant  control  positions. 
DYSIM  training  specialists  staffed  pseudo-pilot  and  remote  controller  positions. 

To  collect  objective  data  for  all  simulation  runs,  each  run  was  extensively  audio  and  video 
recorded.  Guided  post-simulation  discussions  and  questionnaires  were  used  to  obtain  subjective 
data.  Dynamic  workload  probes  were  recorded  at  15-minute  intervals  to  assess  the  level  of 
workload  throughout  a  run. 

Generally,  RVSM  conditions  proved  to  be  more  workload  intensive  than  CVSM  conditions. 
Although  workload  increased,  there  was  no  systematic  increase  in  operational  errors  or 
deviations  with  RVSM  when  compared  to  CVSM.  Both  controller  and  Technical  Observer 
ratings  revealed  that  interval  and  post-run  workload  ratings  were  either  equal  or  higher  for 
RVSM  under  contingency/emergency  (RVSM-E)  conditions  when  compared  to  normal  RVSM 
conditions.  The  operational  errors  analysis  revealed  the  same  trend;  more  errors  were  reported 
under  RVSM-E  conditions.  Therefore,  guidelines  to  handle  potential  complications,  such  as 
radar  outages  and  bad  weather,  need  to  be  developed  before  RVSM  can  be  safely  implemented. 

The  results  of  the  simulation  generally  indicate  that  RVSM  implementation  is  feasible  in  the 
Western  Atlantic  Track  Route  System  region.  However,  controllers  expressed  concerns  about 
safety  with  maintaining  separation  and  transitioning  aircraft  to  and  from  RVSM  altitudes. 
Controllers  also  pointed  out  that  non-RVSM  participating  aircraft  may  present  a  problem  at 
crossing  fixes  because  they  will  have  to  enforce  the  2000-ft  separation  minima.  Controllers 
made  the  following  suggestions  to  enable  RVSM  to  be  implemented  more  efficiently;  (a)  all 
aircraft  operating  in  a  particular  airspace  should  be  RVSM  equipped,  (b)  the  transition  airspace 
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should  be  increased,  and  (c)  controllers  should  have  access  to  more  reliable  radar.  However, 
most  controllers  indicated  their  comfort  level  would  increase  with  increased  exposure  to  RVSM 
and,  therefore,  additional  procedures  and  equipment  may  not  be  necessary. 
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1.  Introduction 


Reduced  Vertical  Separation  Minima  (RVSM)  is  an  approved  International  Civil  Aviation 
Organization  (ICAO)  concept  to  reduce  aircraft  vertical  separation  from  the  Conventional 
Vertical  Separation  Minima  (CVSM)  i  of  2000  feet  to  1000  feet,  between  flight  levels  (FLs)  290 
and  410,  within  a  designated  portion  of  the  North  Atlantic  (NAT)  Region.  An  RVSM-approved 
aircraft  must  have  at  least  two  independent  altitude  measurement  systems,  one  secondary 
surveillance  radar  altitude  reporting  transponder,  one  altitude  alert  system,  and  an  automatic 
altitude  control  system.^ 

The  technical  feasibility  and  cost  benefits  of  establishing  RVSM  in  the  NAT  Region  have  been 
the  subject  of  many  studies  conducted  by  affected  ICAO  member  states.  As  a  result  of  these 
studies,  ICAO  is  planning  for  the  implementation  of  reduced  minimums  in  the  Minimum 
Navigation  Performance  Specification  (MNPS)  portion  of  the  NAT  Region  in  January  1998.  The 
verification  trials  were  originally  scheduled  to  begin  in  January  1997;  however,  the  trials  began 
in  March  1997  with  a  limited  FL  range  of  330  to  370. 

With  the  support  of  the  International  Air  Transport  Association  (lATA),  RVSM  studies  were 
continued  to  include  the  portion  of  the  Western  Atlantic  Track  Route  System  (WATRS)^  area 
under  U.S.  control.  Originally,  it  was  requested  that  the  WATRS  implementation  be  congruent 
with  RVSM  implementation  in  the  MNPS  airspace.  However,  WATRS  implementation  is  not 
scheduled  until  at  least  1998  in  the  U.S.  portion  of  the  airspace  and  as  late  as  1999  or  2000  in  the 
Caribbean  and  South  American  regions. 

In  the  United  States,  the  Federal  Aviation  Administration  (FAA)  conducted  a  series  of  Air 
Traffic  Control  (ATC)  simulations  to  assist  Air  Traffic  organizations  in  identifying  and  defining 
the  requirements  for  implementing  RVSM  in  both  the  MNPS  and  WATRS  airspace. 

The  RVSM  simulation  described  in  this  plan  was  conducted  under  the  auspices  of  the  FAA 
National  Simulation  Capability  (NSC)  Program.  NSC  relied  heavily  on  the  expertise  of 
controllers  and  staff  from  the  Miami  Air  Route  Traffic  Control  Center  [ARTCC]  (ZMA).  NSC 
also  relied  on  the  expertise  of  the  following  organizations:  Air  Traffic  Rules  and  Procedures 
Service  (ATP-100),  Air  Traffic  System  Management  (ATM-100),  Air  Traffic  Plans  and 
Requirements  Service  (ATR-300),  Flight  Standards  (AFS-400),  Program  Analysis  and 
Operations  Research  (ASD-400),  Integrated  Product  Team  for  Oceanic  (AUA-600),  the  Human 
Factors  Branch  (ACT-530),  the  Simulation  and  Systems  Integration  Branch  (ACT-540),'*  the 
Aviation  System  Analysis  and  Modeling  Branch  (ACT-520),  and  the  Washington  Consultant 
Group  (WCG). 


*CVSM  -  2,000  ft  vertical  separation  minimum  (VSM)  above  FL  290  up  to  FL  600,  inclusive. 

^Refer  to  the  Interim  Guidance  Material  on  the  Approval  of  Operations/Aircraft  for  RVSM  Operations. 
^WATRS  is  ME  of  27  33N  77  OOW  -  27  OON  77  OOW  -  20  23N  60  27W. 

'*  Formally  ACD-350,  Simulation  and  Human  Factors  Branch. 
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RVSM  Phase  in  simulations  were  designed  to  measure  the  effects  of  changes  in  standard 
operating  procedures  on  controller  workload  in  ZMA  radar  sectors  R60,  R62,  R63,  and  Rl. 

Phase  in  investigated 

a.  changes  in  controller  workload  levels  as  impacted  by  RVSM  operations; 

b.  operational  issues  associated  with  RVSM  operations  in  R60,  R62,  R63,  and  Rl ; 

c.  operational  difficulties  associated  with  controllers’  ability  to  transition  aircraft  from  and 
to  RVSM  and  to  and  from  CVSM  within  radar  coverage;  and 

d.  other  issues  related  to  reverting  to  and  from  RVSM  and  from  and  to  CVSM. 

1.1  Background 

In  the  late  1950s,  a  need  was  identified  to  increase  the  prescribed  VSM  from  300  m  (1,000  ft) 
due  to  the  inaccuracy  of  pressure-sensing  barometric  altimeters  as  altitudes  increased.  In  1960, 
FL  290  was  selected  as  the  vertical  limit  for  the  300  m  VSM,  and  a  600  m  (2,000  ft)  VSM  was 
established  for  aircraft  operating  above  FL  290.  This  vertical  limit  was  chosen  based  on  the 
operational  ceiling  of  aircraft  at  that  time.  In  1966,  although  FL  290  was  already  established  as 
the  vertical  changeover  level  on  a  global  basis,  consideration  was  already  being  given  to  the 
application  of  RVSM  above  FL  290  on  a  regional  basis.  Consequently,  ICAO  provisions  stated 
that  RVSM  could  be  applied  under  specific  conditions  and  within  designated  portions  of 
airspace.  To  support  this  provision,  ICAO  recognized  that  a  thorough  assessment  of  the  risk 
associated  with  reducing  the  VSM  would  be  required. 

In  1 980,  the  ICAO  Review  of  the  General  Concept  of  Separation  Panel  (RGCSP)  concluded  that 
the  potential  benefits  of  reducing  vertical  separation  above  FL  290  to  FL  410  outweighed  the  cost 
and  time  involved.  Member  states  were  encouraged  to  conduct  the  necessary  evaluations.  In 
1982,  the  RGCSP  coordinated  studies  to  evaluate  reducing  the  VSM  above  FL  290.  In 
December  1988,  the  RGCSP  reviewed  the  results  of  studies  carried  out  by  Canada,  Japan,  the 
member  states  of  EUROCONTROL  (France,  the  Federal  Republic  of  Germany,  the  Netherlands, 
the  United  Kingdom),  the  Union  of  Soviet  Socialist  Republics,  and  the  United  States. 

-9 

Using  a  Target  Level  of  Safety  (TLS)  of  2.5  x  10  fatal  accidents  per  aircraft  flight  hour,  the 
RGCSP  concluded  that  a  300-m  VSM  above  FL  290  was  technically  feasible.  Contemporary 
height-sensing  systems  can  be  built,  maintained,  and  operated  so  that  the  expected  performance 
is  consistent  with  the  safe  implementation  and  use  of  a  300-m  VSM  above  FL  290.  In  reaching 
this  conclusion,  the  panel  also  fuund  that  it  would  be  necessary  to  establish 

a.  air-worthiness  performance  requirements  embodied  in  a  comprehensive  Minimum 
Aircraft  System  Performance  Specification  (MASPS)  for  all  aircraft  utilizing  the  reduced 
separation, 

b.  new  RVSM  operational  procedures,  and 

c.  a  comprehensive  means  of  monitoring  the  safe  operation  of  the  system. 

The  RGCSP  identified  the  NAT  Region  as  an  area  where  early  implementation  of  RVSM  was 
possible  because  of  its  traffic  patterns  and  aircraft  equipment  requirements.  On  this  basis,  and  in 
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view  of  the  substantial  benefits,  the  NATSPG,  at  its  26th  meeting,  agreed  to  carry  out  studies 
aimed  at  achieving  early  implementation  of  RVSM  in  the  NAT  Region.  Worldwide  and  regional 
provisions  concerning  the  implementation  of  RVSM  were  finalized  for  application  in  November 
1992.5 

Initially,  RVSM  implementation  was  to  be  within  the  MNPS  airspace  (refer  to  Figure  1)  of  the 
ICAO  NAT  Region;  thus  RVSM-related  research  focused  only  on  that  airspace.  A  two-phased 
simulation  was  designed  to  investigate  and  measure  the  effects  of  RVSM  implementation  and  the 
associated  transitions  in  the  MNPS  airspace  and  in  the  radar  airspace  adjacent  to  the  MNPS 
airspace.  Both  areas  are  under  the  control  of  the  New  York  ARTCC  (ZNY).  With  the  support  of 
the  lATA,  it  was  requested  that  RVSM  studies  be  continued  to  include  the  portion  of  the 
WATRS  area  under  U.S.  control.  lATA  felt  that  users  would  further  benefit  from  RVSM 
implementation  in  that  area.  Therefore,  a  third  phase  of  the  simulation  was  added  to  analyze  the 
effects  of  transitioning  RVSM-approved  aircraft  in  the  airspace  that  is  adjacent  to  and  south  of 
the  MNPS  airspace  (i.e.,  the  WATRS  portion  of  the  NAT  Region  under  ZMA  control). 

RVSM  Phase  I  simulation  was  completed  in  January  1994.  It  was  a  study  of  the  transition  of 
westbound  RVSM-approved  aircraft  from  RVSM  to  CVSM  before  leaving  RVSM/MNPS 
airspace  (refer  to  Figure  1).  There  was  an  exception  for  aircraft  that  entered  radar  coverage 
adjacent  to  RVSM/MNPS  airspace  or  Canadian  airspace  (this  was  either  MNPS  or  radar 
coverage).  RVSM  increased  the  amount  of  available  altitudes  thus  providing  the  controller  with 
greater  flexibility  for  managing  traffic.  However,  simulation  results  indicated  that  controllers 
operating  under  RVSM  experienced  additional  coordination  with  adjacent  sectors  and  facilities, 
increased  traffic  scanning  times,  and  an  increased  need  for  information  regarding  when  an 
aircraft  could  climb.  As  a  result,  interval  increases  in  controller  workload  occurred  under  RVSM 
traffic  conditions  when  compared  to  CVSM  conditions.  However,  the  overall  controller 
workload  did  not  increase.  Results  indicated  that  transition  in  non-radar  sectors  D71  and  D72 
was  feasible,  although  controllers  recommended  that  transitions  take  place  in  radar-controlled 
airspace  (Seeger,  1995). 

RVSM  Phase  n  simulation  was  completed  in  September  1994.  It  was  a  study  of  RVSM 
transitions  in  domestic  oceanic  airspace  (refer  to  Figure  2).  RVSM-approved  aircraft  were 
permitted  to  exit  or  enter  the  MNPS  airspace  from  the  west  using  RVSM  rules.  The  associated 
transitions  would  occur  within  the  adjacent  radar-controlled  sectors  R65  and  R86  under  New 
York  ARTCC  control.  Results  indicated  that  RVSM  was  instrumental  in  reducing  controller 
workload  when  a  majority  of  the  traffic  traveled  eastbound.  However,  controllers  reported  some 
concerns  including:  separating  RVSM-equipped  and  non-RVSM-equipped  aircraft,  difficulty 
maintaining  data  block  separation  during  RVSM,  and  the  possibility  of  aircraft  flying  into  CVSM 
airspace  at  an  RVSM  altitude  due  to  a  temporary  lack  of  communication  (Seeger  &  Kopardekar, 
1996). 


5  Manual  on  implementation  of  a  300  m  (1000  ft.)  VSM  between  FL  290  and  FL  410  inclusive  is  ICAO  Doc.  No.  9574- 
AN/934,  dated  1992. 
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Figure  1.  RVSM/MNPS  airspace  used  for  Phase  I  simulations. 


Figure  2.  RVSM  transition  airspace  used  for  Phase  n  simulations. 
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RVSM  Phase  m  simulation  was  completed  in  October  1995.  This  phase  focused  on  issues 
arising  from  transitioning  RVSM-approved  aircraft  in  the  WATRS  portion  of  the  NAT  Region 
under  ZMA  control  (refer  to  Figure  3).  Aircraft  eventually  traversing  through  WATRS  airspace 
would  be  permitted  to  exit  or  enter  the  MNPS  airspace  using  RVSM  rules,  and  the  associated 
transition  would  occur  within  radar  coverage  provided  by  ZMA. 


wso*  wes” 


Figure  3.  RVSM  transition  airspace  used  for  Phase  ni  simulations. 


During  the  planning  stage  of  Phase  HI,  ZMA,  in  conjunction  with  ZNY,  initiated  steps  to 
reconfigure  the  ZMA  portion  of  WATRS  airspace.  Refer  to  Appendix  A  for  close-ups  of  the 
individual  sectors  used  in  the  simulation.  These  changes  were  not  put  into  effect  until  after  the 
simulation  was  completed.  The  reconfiguration  was  assumed  to  result  in  the  alleviation  of 
controller  workload  by  reducing  the  quantity  of  aircraft  that  would  be  vectored.  The 
reconfiguration  was  not  dependent  on  the  use  of  RVSM.  ZMA  personnel  conveyed  that  various 
other  projects  were  also  under  way  to  reconfigure  their  airspace. 

Even  though  ZMA  personnel  anticipated  changes  in  the  airspace,  they  felt  that  it  was  an 
appropriate  time  to  conduct  the  simulation.  The  Airspace  and  Procedures  Branch,  ZMA-530, 
and  the  NSC  program  agreed  to  the  following: 
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a.  Studying  RVSM  under  the  configuration  at  the  time  of  the  simulation  would  provide 
valuable  information  that  would  be  applicable  regardless  of  airspace  configuration. 

b.  The  simulation  would  provide  data  that  would  be  beneficial  when  making  near-term 
airspace  design  and  procedural  changes  while  considering  the  future  implementation  of 
RVSM. 

c.  Rescheduling  the  Dynamic  Simulation  (DYSIM)  Laboratory  and  all  the  participants  and 
support  personnel  would  have  been  very  difficult. 

d.  Contract  dollars  were  already  obligated  to  support  the  simulation  and  funding  was 
available. 

The  decision  to  conduct  the  simulation  on  schedule  was  financially  responsible. 

1.2  Purpose 

The  long  range  forecast  for  the  NAT  Region  estimates  that  air  traffic  will  double  by  the  year 
2010^.  The  reduction  of  vertical  separation  in  the  MNPS  airspace,  NAT  Region,  between  FL  290 
and  FL  410  inclusive,  would  theoretically  accommodate  such  a  projected  increase  in  air  traffic. 
This  enhancement  in  system  capacity  would  provide  for  a  more  efficient  use  of  the  available 
airspace  and  result  in  significant  improvements  in  flight  economy. 

The  most  difficult  problem  with  operating  under  RVSM  in  the  MNPS  airspace  will  probably  be 
the  transition  of  aircraft  to  CVSM.  Additionally,  the  procedures  for  transition  may  differ  based 
upon  the  geographical  restrictions.  Therefore,  ATC  procedures  for  the  various  RVSM  transition 
areas  within  the  NAT  Region  and  adjacent  ICAO  Regions  need  to  be  defined  prior  to  the 
implementation  of  RVSM.  Accordingly,  this  simulation  evaluated  procedures  used  by 
controllers  when  transitioning  aircraft  to  and  from  RVSM  and  from  and  to  CVSM. 

This  simulation  had  two  objectives.  The  first  objective  was  to  study  the  feasibility  of 
transitioning  RVSM  aircraft  in  the  identified  geographical  areas  and  to  identify  problems 
associated  with  that  transition.  The  second  objective  was  to  address  changes  in  controller 
workload  caused  by  potential  increased  flight  operations.  Part  of  the  second  objective  was  to 
study  the  impact  of  weather-related  problems  and  contingencies  that  could  cause  aircraft 
deviations  on  controller  workload. 

The  RVSM  simulation  and  associated  activities  were  designed  to  provide  Air  Traffic 
organizations,  especially  the  International  Procedures  Branch  (ATP- 140),  with  the  vital,  human 
performance  information  needed  to  define  RVSM  implementation  procedures.  This  simulation 
represented  a  critical  step  by  Air  Traffic  Service  (ATS)  in  assessing  current  and  projected  ZMA 
oceanic  ATC  system  capabilities.  The  results  of  this  study  were  closely  coordinated  and  shared 
with  all  NAT  ATC  provider  states  to  help  facilitate  the  development  of  a  unified  implementation 
plan. 


^Agenda  Item  2,  Working  Paper  131  presented  at  the  Limited  North  Atlantic  (COM/MET/RAC)  Regional  Air  Navigation 
Meeting  held  in  Cascais,  Portugal,  in  November  1992. 
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2.  Method 


RVSM  Phase  HI  was  conducted  in  the  ZMA  DYSIM  Laboratory  from  October  16  through  27, 
1995.  The  simulation  occurred  between  19:00  Zulu  and  02:00  Zulu  for  10  working  days, 
exclusive  of  the  weekends.  The  immediate  physical  environment  realistically  simulated  the 
ZMA  radar  sectors  R60,  R62,  R63,  and  Rl,  including  the  available  control  equipment  and 
communication  interfaces. 

This  project  incorporated  real-time  ATC  simulations  designed  to  evaluate  workload  when 
controllers  provided  separation  and  other  ATC  services  in  designated  domestic  oceanic  radar- 
controlled  transition  airspace.  Phase  HI  allowed  for  the  transition  of  RVSM-approved  aircraft  to 
and  from  RVSM  and  from  and  to  CVSM  standard  in  the  WATRS  portion  of  ZMA  airspace.  The 
transition  occurred  when  RVSM-approved  aircraft  entered  the  mostly  radar-controlled  sectors 
R60,  R63,  or  Rl. 

The  RVSM  simulation  adhered  to  the  following  international  guidelines  as  a  basis  for  developing 
each  simulation  scenario: 

a.  RVSM  was  affected  coincident  with  MNPS  airspace  and  in  defined  transition  areas. 

b.  The  transition  to  and  from  reduced  CVSM  was  affected  in  transition  areas. 

c.  The  transition  areas 

1 .  were  defined  as  Class  A  airspace;  accordingly,  aircraft  proceeding  to  and  from  MNPS 
airspace  were  authorized  to  transition  to  and  from  1000  ft  VSM; 

2.  were  contained  within  horizontal  limits  determined  by  provider  states,  either 
individually  or  in  conjunction; 

3.  were  adjacent  to,  overlapping  with,  or  contained  within  MNPS  airspace; 

4.  were  within  radar  coverage  using  direct  controller/pilot  communications  wherever 
practical;  and 

5.  were  contained  within  the  vertical  limits  of  FL  290  to  FL  410,  inclusive. 

d.  When  operating  within  transition  areas,  RVSM  was  applied  between  aircraft  approved  for 
such  operations  (refer  to  Section  1)  when  transiting  to  and  from  MNPS  airspace. 

In  addition,  NSC  worked  with  ZMA  personnel  and  DYSIM  training  specialists  to  further  develop 
experimental  guidelines.  These  included  sector  configurations,  run  time,  controller  staffing, 
number  of  aircraft  per  scenario  (which  was  sometimes  limited  by  DYSIM  operator  capabilities), 
and  contingencies. 

Each  simulation  parameter  was  designed  to  enable  valid  comparisons  between  current  CVSM 
operations  and  planned  RVSM  operations.  RVSM  Phase  HI  investigated  the  following  and  relied 
on  responses  from  the  associated  questions  for  analysis  of 

a.  changes  in  controller  workload  levels  as  impacted  by  RVSM  operations, 

b.  operational  issues  associated  with  RVSM  operations. 
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c.  operational  difficulties  associated  with  controllers’  ability  to  transition  aircraft  to  and 
from  RVSM  within  radar  coverage,  and 

d.  other  issues  related  to  reverting  from  RVSM  rules  to  CVSM  rules  such  as  the 

1 .  most  frequently  occurring  questions  or  problems,  and 

2.  occurrences  under  special  scripted  events  (unreliable  radar  coverage  and  adverse 
weather  conditions). 

2.1  Participants 

Fifty-six  currently  certified  controllers  from  ZMA  participated  in  this  simulation.  Thirty-eight 
controllers  staffed  control  positions  and  18  controllers  participated  as  technical  observers  (T/Os). 

Five  DYSM  training  specialists  from  the  Washington  Consulting  Group  (WCG)  staffed  two 
pseudo-pilot  positions.  Another  WCG  specialist  staffed  a  remote  controller  position.  ACT-530 
and  ACT-540  provided  five  additional  support  personnel  from  the  FAA  William  J.  Hughes 
Technical  Center. 

2.1.1  Background  Survey 

A  background  survey  (refer  to  Appendix  B)  was  conducted  to  collect  basic  demographic  data  and 
information  about  the  participants’  opinions  regarding  high  workload  situations.  Data  from  this 
survey  were  used  as  a  baseline  for  comparison  with  survey  responses  collected  throughout  the 
simulation.  Appendix  C  contains  a  summary  of  background  information. 

The  average  age  of  the  controllers  and  T/Os  was  33.  The  controllers  had  an  average  of  6.6  years 
of  ATC  experience  and  an  average  of  4.3  of  those  years  at  full  performance  level  (FPL).  The 
T/Os  had  more  experience  overall  with  an  average  of  8.6  years  of  ATC  experience  and  an 
average  of  6. 1  of  those  years  at  FPL.  Fifteen  of  the  1 8  T/Os  also  had  previous  experience 
training  controllers.  Forty-five  of  the  56  participants  had  controlled  traffic  for  ZMA  only. 

Figures  4,  5,  6,  and  7  depict  questionnaire  responses. 

Question  6  of  the  Background  Information  Form  asked  participants  to  list  factors  affecting  safe 
and  expeditious  traffic  flow  in  their  current  ATC  environment.  Figures  4  and  6  display  the 
frequency  of  these  responses.  Fifteen  participants  responded  to  questions  for  Area  2,  and  41 
participants  responded  to  questions  for  Area  4.  “Comm.  Failure”  consists  of  frequency  problems 
and  outages  and  aircraft  radio  failure.  “Control  Issues”  encompasses  concerns  such  as  lack  of 
available  altitudes,  excessive  coordination,  and  aircraft  failing  to  report  over  fixes.  “Foreign 
Language”  refers  to  difficulties  communicating  with  foreign  pilots  and  facilities.  “Other” 
includes  anything  that  does  not  fit  into  the  previously  mentioned  categories  such  as  military 
operations,  emergencies,  and  hijackings.  “Radar  &  Equipment”  includes  radar  outages,  blind 
spots  in  the  radar  coverage,  equipment  failure,  and  navigation  aids.  For  Area  2,  Control  Issues 
was  the  most  frequently  listed  factor.  In  Area  4,  more  participants  listed  Radar  &  Equipment  as  a 
factor  affecting  safe  and  expeditious  traffic  flow. 
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Figure  4.  Factors  affecting  safe  and  expeditious  traffic  flow  in  Area  2. 
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Figure  5.  Mean  rating  of  factors  contributing  to  high  workload  in  Area  2. 


9 


AREA  4 


Factors 


Figure  6.  Factors  affecting  safe  and  expeditious  traffic  flow  in  Area  4. 

AREA  4 


Aircraft  Coord.  Coord.  Printer  Comm.  System 

Factors 


Figure  7.  Mean  rating  of  factors  contributing  to  high  workload  in  Area  4. 

On  a  scale  from  1  to  8,  controllers  and  T/Os  rated  seven  aspects  of  ATC  for  their  contribution  to 
high  workload  levels  in  the  current  ATC  environment.  Figures  5  and  7  show  the  mean  rating  for 
each  factor  by  area.  For  Area  2,  a  mix  of  aircraft  performance/characteristics  had  the  highest 
average  rating  (M  =  4.4,  SD  =  2.0).  Coordination  with  foreign  facilities  had  the  highest  average 
rating  for  contribution  to  workload  in  Area  4  (M  =  5.9,  SD  =  1.5). 
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2.2  Equipment/Configuration 
2.2.1  DYSIM  Laboratory 


The  ZMA  DYSIM  Laboratory  provided  controllers  with  a  realistic  simulated  radar  environment. 
All  the  planned  scenarios  were  stored  as  files  on  a  DYSIM  tape.  Simulated  (SIM)  radar  targets 
were  generated  using  the  Simulation  Start  action  on  the  aircraft  in  the  DYSIM  files.  A  SIM 
target  represents  the  radar  trails  of  a  maneuvering  aircraft. 

The  Multiple  Radar  Processing  function  of  the  National  Airspace  System  (NAS)  generated  and 
processed  primary  and  beacon  radar  data  for  each  SIM  target  in  a  manner  similar  to  normal  radar 
data.  The  SIM  flight  data  block  contained  the  SIM  flight  identification,  magnetic  heading, 
beacon  code,  and  altitude.  The  position  of  the  SIM  radar  data  was  automatically  updated 
approximately  every  10  seconds. 

The  SIM  radar  targets  maneuvered  automatically  based  on  route  segments  from  a  flight  plan  and 
by  operator  inputs  into  a  Computer  Readout  Display  (CRD)  to  depict  actual  aircraft  operations. 
The  CRD  allowed  the  pilot  to  alter  10  aspects  of  an  aircraft  flight  (altitude,  routing,  rate  of  climb, 
etc.).  The  result  was  a  totally  simulated  flight  that  could  exercise  almost  all  functions  as  if  there 
was  a  paired  flight  plan  and  flight  data  processing  with  flight  progress  strip  preparation 
capability. 

2.2.2  Voice  Communication  System 

The  voice  communication  was  a  Robert  Thomas  Smith  (RTS)  Systems  Model  CS9500  Digital 
Intercom  System.  The  RTS  CS9500  is  a  programmable  intercommunication  system  that 
maintains  high  quality  speech  characteristics  utilizing  a  4-wire,  central,  non-blocking  matrix 
design.  Voice  communication  key  panels  were  attached  to  the  matrix  via  RJ-1 1  cabling.  An 
MS-DOS-based  package  called  CSEdit,  operating  on  a  486  laptop  PC  connected  to  the  matrix 
through  the  serial  communication  port,  provided  system  programming. 

The  controllers  used  voice  communication  key  panels  that  provided  communications 
functionality  comparable  to  the  control  floor  exclusive  of  a  shout  line.  Party  lines  were  used  to 
emulate  radio  frequencies  between  controllers  and  pilots.  Point-to-point  communications  were 
used  for  communication  between  adjacent  sectors  and  centers.  The  matrix  was  programmed  to 
meet  the  specifications  required  by  the  various  simulation  configurations. 

2.2.3  Audio  and  Video  Recording  Rack 

The  audio  and  video  equipment  and  the  voice  communication  matrix  were  mounted  in  two 
mobile  equipment  racks.  Each  rack  was  specifically  designed  to  protect  sensitive  equipment 
during  shipping  and  provided  work  surfaces  during  the  simulation. 

Video  data  collection  consisted  of  three  separate  camera  views  recorded  from  black  and  white, 
low-light  micro-cameras.  Two  cameras  recorded  each  sector  individually,  and  the  third  recorded 
an  overall  view  of  the  controller  plan  view  display  (PVD)  row.  The  video  was  recorded  in  Super 
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VHS  format  on  2-hour  tapes  stamped  with  National  Television  System  Committee  linear  time 
code  for  synchronous  playback  purposes. 


Eight  separate  audio  signals  were  recorded  from  wireless  microphones  worn  by  each  controller 
and  four  audio  channels  recorded  directly  from  the  communication  system.  The  audio  signals 
were  mixed  on  a  Tascam  M2516  audio  mixing  board  and  recorded  on  the  Hi-Fi  audio  channels 
of  the  videotapes  according  to  the  corresponding  camera  views.  They  were  also  recorded 
separately  onto  a  16-track  1/2”  audiotape  for  backup  purposes  or  to  allow  audio  signal  isolation. 

2.3  Scenarios 


A  DYSIM  training  specialist  with  previous  ZMA  ATC  experience  developed  each  scenario. 
These  scenarios  were  based  on  flight  plans  extracted  from  Data  Analysis  and  Reduction  Tool 
(DART)  runs  of  System  Analysis  and  Recording  (SAR)  tapes  from  both  ZMA  and  ZNY. 

Table  1  describes  the  1 1  one-hour  scenarios  run  during  Phase  HI.  Scenarios  varied  according  to 
the  following  parameters:  separation  minima,  sector  configuration,  number  of  controllers  and 
aircraft  at  each  sector,  and  contingencies.  Contingency  problems  consisted  of  simulated  radar 
outages  (Nassau  radar  in  R60  and  Grand  Turk  radar  in  R63)  or  adverse  weather  conditions.  Each 
scenario  was  named  with  a  number  and  a  letter.  Scenarios  with  the  same  number  had  the  same 
number  of  aircraft,  control  positions,  and  sector  configuration.  A  C,  R,  or  E  follows  these 
numbers  indicating  whether  the  separation  was  CVSM,  RVSM,  or  RVSM  with  eontingencies 
(RVSM-E),  respectively. 


Table  1.  Summary  of  Scenarios 


Scenario 

Name 

Separation 

Minima 

Sector  on 
PVD93 

PVD  93 
Control 
Positions 

Sector  on 
PVD  94 

PVD  94 
Control 
Positions 

Contingency 

IR 

RVSM 

R63 

R,D 

R60 

R,D 

- 

1C 

CVSM 

R,D 

R60 

R,  D 

> 

IE 

RVSM 

R63 

R,D 

R60 

R,D 

Weather/Radar 

2R 

RVSM 

R62/R63 

R,D 

R60 

R,D 

- 

2C 

CVSM 

R62/R63 

R,D 

R,D 

- 

2E 

RVSM 

R62/R63 

R,D 

R,  D 

Weather/Radar 

3R 

RVSM 

R60/R62/R63 

R,D 

R1 

R,D 

- 

3C 

CVSM 

R60/R62/R63 

R,D 

R1 

R,D 

- 

4R 

RVSM 

R62/R63 

R,D,A 

- 

- 

- 

4C 

CVSM 

R62/R63 

R,D,A 

- 

- 

- 

4E 

RVSM 

R62/R63 

R,  D,  A 

- 

- 

Weather/Radar 
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All  scenarios  were  run  in  the  DYSIM  Laboratory  using  PVDs  93  and  94.  The  sectors  in  question 
(e.g.,  Rl,  R60,  R62,  and  R63)  were  configured  on  each  PVD  to  match  the  configuration  on  the 
control  floor. 

ZMA  sector  density  analysis  reports  and  input  from  Area  2  and  Area  4  specialists  determined  the 
number  of  aircraft  in  each  scenario.  DYSIM  operator  input  capabilities  limited  the  maximum 
number  of  aircraft  per  scenario.  It  was  determined  that  the  number  of  aircraft  required  to 
represent  a  busy  day  was 

•  35-40,  if  all  three  positions  [i.e..  Radar  (R),  Hand-Off  (D),  and  Associate  (A)]  were 
staffed;  and 

•  25-30,  if  two  positions  (i.e.,  R  and  D)  were  staffed. 


Between  10%  and  20%  of  the  aircraft  in  each  scenario  filed  flight  plans  at  RVSM  altitudes.  This 
percentage  was  based  on  aircraft  that  would  be  eligible  for  RVSM  at  the  time  of  the  simulation. 
Table  2  lists  the  number  of  scripted  pilot  events  and  the  number  of  aircraft  in  each  scenario. 


Table  2.  Frequency  of  Pilot  Events 


PVD“ 

93  94 

93  94 

Pilot  Events 

Number  of  Aircraft 

Scenario 

IR 

6 

6 

27 

27 

1C 

6  or  7 

6 

27 

27 

IE 

6 

6 

29 

27 

2R 

6 

6 

27 

26 

2C 

6  or  7 

6 

27 

26 

2E 

6 

6 

29 

25 

3R 

6  or  7 

6 

27 

28 

3C 

6  or  7 

6 

27 

28 

4R 

6  or  7 

5  or  6 

39 

- 

4C 

6  or  8 

6 

39 

- 

4E 

6  or  7 

6 

39 

- 

1  ^  Refer  to  Table  1  for  description  of  PVD  93  and  94  | 

2.4  Experimental  Procedures 

Table  3  lists  the  daily  schedule  for  the  10  days  of  simulation  including  scenario  name,  date,  run 
time,  and  contingency.  Five  scenarios  were  run  daily  except  for  the  last  day,  which  included 
four.  Scenarios  were  assigned  randomly  except  for  the  last  two  runs  of  the  first  5  days.  Due  to 
sector  configurations  on  the  control  room  floor,  it  was  imperative  that  Scenarios  3C  and  3R  be 
run  at  the  end  of  the  evening.  Figure  8  indicates  the  laboratory  configuration  for  Scenarios  1 , 2, 
and  3.  Figure  9  indicates  the  laboratory  configuration  for  Scenario  4. 


13 


Table  3.  Scenario  Schedule 


Day 

Run 

Date 

ZULU  Time 

1 

IR 

10/16/95 

19:15-20:15 

2 

2R 

10/17/95 

19:12-20:13 

3 

4R 

10/18/95 

19:05  -  20:05 

4 

4C 

10/19/95 

19:02  -  20:02 

5 

IR 

10/20/95 

19:00  -  20:00 

6 

2R 

10/23/95 

19:04  -  20:03 

7 

2C 

10/24/95 

19:02  -  20:02 

8 

1C 

10/25/95 

18:59-19:59 

9 

IR 

10/26/95 

18:44-19:44 

10 

1C 

10/27/95 

18:58-19:58 

Run 

ZULU  Time 


4E  (Radar) 
20:24-21:25 


4R 

20:20-21:19 


1C 

20:15-21:15 


2C 

20:21-21:21 


2R 

20:20-21:20 


IR 

20:14-21:14 


2E  (Radar) 
19:59  -  20:29 


2C 

20:10-21:10 


Run 

ZULU  Time 

Run 

ZULU  Time 

Run 

ZULU  Time 

IE  (Radar) 
21:52-22:49 

3C 

00:05-01:03 

3R 

01:36-02:19 

2E  (Weather) 
21:44-22:44 

3R 

23:47  -  00:46 

3C 

01:02-01:59 

4C 

21:40-22:40 

3R 

23:46  -  00:42 

3C 

01:00-02:00 

4E  (Radar) 
21:33-22:33 

3C 

23:55  -  00:52 

3R 

01:15-02:02 

IE  (Radar) 
21:25-22:25 

3C 

23:37  -  00:36 

3R 

00:47  -  01:45 

4E  (Radar) 
21:41-22:41 

4C 

23:43  -  00:43 

4R 

01:00  -  02:00 

4R 

21:44-22:44 

4E  (Weather) 
23:50  -  00:50 

4C 

01:07  -  02:07 

IE  (Weather) 
21:32-22:31 

4C 

23:38  -  00:38 

4R 

00:55-01:55 

1C 

21:15-22:15 

2C 

23:11-00:11 

2R 

00:25-01:25 

IR 

21:25-22:25 

2R 

23:12-00:12 

a 

'M. 


mmn 


a 


^  % 


Figure  8.  DYSIM  Laboratory  configuration  for  Scenarios  1,  2,  and  3. 


Figure  9.  DYSIM  Laboratory  configuration  for  Scenario  4. 

For  one-half  hour  before  the  start  of  the  first  run  of  the  day,  participants  received  a  briefing  and 
became  familiar  with  the  communications  equipment.  During  the  briefing,  controllers  were 
instructed  to  apply  all  the  current  ATC  standards  with  the  addition  of  RVSM.  For  purposes  of 
the  simulation,  controllers  were  also  instructed  to  adhere  to  the  following  guidelines  set  forth  by 
ZMA-530: 

a.  RVSM  transitions  must  take  place  in  the  WATRS  portion  of  a  transition  sector  except  for 
Rl. 

b.  Rl  must  send  aircraft  to  ZNY  at  RVSM  altitudes  but  will  not  receive  them  from  ZNY  at 
RVSM  altitudes. 

c.  Sectors  Rl ,  R60,  and  R63  are  the  only  RVSM  transition  sectors. 

d.  Aircraft  flying  in  San  Juan  airspace  do  not  need  to  be  transitioned  before  entering  and 
exiting  ZMA  airspace  during  the  simulations.  (It  is  assumed  that  San  Juan  will  also  be 
using  RVSM). 

In  addition,  controllers  were  instructed  to  allow  an  aircraft  to  operate  at  RVSM  altitudes  if 

a.  it  was  a  B747,  B74F,  B767,  B757  (American  Trans  Air  or  Iceland  Air),  DC-10,  MD-1 1, 
L-101 1,  G-2,  G-3,  G-4,  CL-60,  DA-50,  DA-90,  or  HS-25;  and 

b.  it  was  departing  or  arriving  from  Europe,  Africa,  or  Greenland,  or  it  was  flying  airway 
R759,  A554,  G431,  R763,  B646,  G437,  A699,  or  A700  from  NY. 

Because  the  exact  fleet  of  aircraft  that  will  be  RVSM  eligible  has  not  yet  been  determined,  these 
criteria  were  established  from  a  list  of  RVSM-approved  operators  and  aircraft.  Aircraft  were  on 
this  list  if  they  were  approved  by  their  state  of  registry.  For  future  reference  or  updates,  this  list 
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is  maintained  in  a  central  data  base  at  the  United  Kingdom  Civil  Aviation  Authority  in  London, 
England. 

2.4. 1  Controller  Assignment 

A  different  group  of  Area  2  and  Area  4  controllers  were  scheduled  to  participate  in  the 
simulation  each  day.  None  of  the  controllers  worked  traffic  for  the  same  scenario  more  than 
once.  Once  a  controller  was  assigned  to  a  control  position  (i.e.,  R,  D,  or  A)  at  a  particular  sector, 
he  or  she  remained  at  that  control  position  and  sector  for  all  variations  of  that  scenario  (i.e., 
CVSM,  RVSM,  RVSM-E).  The  same  was  true  for  the  T/Os. 

The  Area  Supervisors  determined  which  control  position  each  controller  worked  and  which 
controllers  participated  as  T/Os.  For  example.  Area  Supervisors  approved  controller  303 1  to 
work  the  R  position  and  controller  1742  to  be  a  T/0.  On  the  first  day,  each  was  assigned  to 
sector  60  during  Scenario  IR.  They  remained  at  sector  60  observing  and  controlling  the  R 
position  for  Scenarios  1C  and  IE. 

2.4.2  Data  Collection 


All  participants  anonymously  filled  out  the  Background  Information  i^orm  and  Post-Run 
Questionnaire,  reported  interval  workload  values  (refer  to  Appendix  B)*,  and  participated  in 
recorded  debriefing  sessions.  SAR  data  were  recorded  for  each  run.  For  completeness,  each  run 
was  also  audio  and  video  recorded.  Simulation  data  were  collected  via 

a.  automated  recording  of  Host  data  via  SAR  tapes, 

b.  real-time  observations  of  critical  controller  actions  (operational  errors  and  deviations) 
recorded  by  T/Os  throughout  each  simulation  run, 

c.  real-time  interval  workload  ratings^  made  by  the  T/Os, 

d.  real-time  interval  workload  ratings  obtained  by  having  support  personnel  prompt  a  verbal 
report  from  all  controllers  throughout  each  run, 

e.  participant  responses  to  Post-Run  Questionnaires  and  structured  interviews  conducted 
after  each  simulation  run,  and 

f.  support  personnel  tabulations  of  controller  communications.  ‘  * 

Questions  that  required  participant  ratings  were  based  on  an  8-point  scale.  For  workload  ratings, 
1  indicated  very  low  workload  and  8  indicated  very  high  workload.  For  other  ratings,  8  indicated 


^  Due  to  schedule  changes,  some  people  participated  more  than  1  day. 

^  When  an  A  controller  was  present,  an  additional  form  was  provided.  The  form  was  not  included  in  Appendix  B  because 
the  questions  are  the  same. 

^  Four  interval  ratings  obtained,  one  rating  every  1 5  minutes. 

This  procedure,  Air  Traffic  Workload  Input  Technique  (ATWIT),  is  FAA- validated  and  used  for  the  continual 
assessment  of  controller  workload. 

^  ^  A  communication  was  measured  every  time  a  controller  verbalized  a  message  into  his  or  her  head  set. 
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better  performance.  Table  4  defines  each  subjective  measure.  Table  5  defines  the  analyzed  SAR 
messages. 

Some  SAR  data  were  not  recorded  because  of  operator  recording  errors.  These  errors  were  not 
discovered  within  15  days  of  the  recording  date,  therefore,  the  data  were  not  recoverable.  Table  6 
lists  the  scenarios  with  partial  or  no  data.  Partial  SAR  data  were  used  whenever  it  was 
applicable. 


Table  4.  Subjective  Dependent  Measures 


Measurement 

Low  Rating  (  1 ) 

High  Rating  (  8  ) 

Maintenance  of  Separation  -  Adhered  to  separation 
standards  as  specified  in  ATP  71 10.65 

Could  not  maintain 
separation 

Easily  maintained 
separation 

Workload 

Easily  completed  all  tasks 
accurately 

Could  not  complete  all  tasks 

Communication  -  Used  ATP  71 10.65 
specifications 

Rarely  followed  ATP 

7110.65  specifications 

Always  followed  ATP 
7110.65  specifications 

Coordination  -  Provided  complete  and  correct 
information  for  clearances  and  estimates  in  a 
timely  manner 

Frequently  gave  incorrect  or 
incomplete  information 

Always  gave  correct 
information 

Maintenance  of  Situational  Awareness  -  Addressed 
all  aspects  of  airspace  and  aircraft 

Could  not  maintain 

awareness 

Easily  maintained 
awareness 

Priority  of  Actions  -  Issued  control  instructions  in 
prioritized,  structured  manner  as  specified  in  ATP 
7110.65 

Did  not  follow  specified 
priority 

Always  followed  specified 
priority 

Flight  Strip  Management  -  Marked  strips 
accurately  while  performing  other  tasks;  kept  strips 
current 

Could  not  maintain  strips 

Easily  maintained  strips 

Computer  Entry  -  Accurately  input  control 
commands  into  key  panel 

Had  great  difficulty 
inputting  commands 

Easily  and  accurately  input 
commands 
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Table  5.  Objective  SAR  Dependent  Measures 


Message 

Definition 

ACCPT 

A  received/entered  message  has  met  all  NAS  message  checking  criteria 

AM 

Input  to  amend  a  flight  plan 

ERROR 

Generated  when  the  computer  determines  that  a  field  or  combination  of  fields  in  an  input  message 
violates  the  acceptance  criteria  for  that  particular  message 

OF 

Flight  plan  readout  request,  similar  to  a  ‘FR’  entry 

ON 

Accept  handoff,  initiate  handoff,  data  block  offset,  force  data  block,  emergency  airport  display 

QP 

Distance  reference  indicators,  point  out,  request/suppress  data  blocks 

OQ 

Interim  altitude 

OR 

Reported  altitude 

OT 

Assigned  altitude,  coast  track,  track 

ou 

Track  re-route,  route  display 

OX 

Drop  tracks  only 

QZ 

Assigned  altitude,  accept  handoff,  initiate  handoff,  data  block  offset,  force  data  block,  emergency 
airport  display 

Message  returned  when  the  computer  program  determines  an  input  message  to  be  in  error 

!■ 

Requested  the  printing/reprinting  at  the  desired  position  of  one  flight  progress  strip  for  the  specified 
flight 

Table  6.  Scenarios  With  Missing  Data 


Date 

Scenario 

Amount  of  Data 

10/17 

2C 

Partial 

10/17 

2E 

Partial 

10/18 

4R 

None 

10/18 

4E 

Partial 

10/18 

4C 

Partial 

10/18 

3R 

Partial 

10/18 

3C 

Partial 

10/19 

4C 

None 

10/20 

3C 

Partial 

10/27 

2R 

Partial 

2.5  Simulation  Fidelity 

Question  3  of  the  Post-Run  Questionnaire  addressed  simulation  realism.  Controllers  and  T/Os 
rated  three  aspects  of  the  simulation  for  realism:  traffic  flow,  physical  environment,  and  taskload. 
A  significant  difference,  F(4, 298)  =  4.44,  existed  between  ratings  given  for  traffic  flow  for 
scenarios  in  Area  2  (sector  Rl)  compared  to  scenarios  for  Area  4  (sectors  R60,  R62,  R63). 
Controllers  and  T/Os  gave  average  ratings  of  6.28  through  6.41  (SD  =  1.50)  for  the  Area  4 
scenarios.  A  significantly  lower  average  rating  of  5. 10  {SD  =  1 .70)  was  reported  for  Area  2. 
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Reasons  for  lower  ratings  in  Area  2  included  not  enough  departures,  high  volume  of  non-radar 
traffic,  and  high  volume  of  crossing  traffic.  It  should  be  noted  that  the  simulation  was  designed 
to  focus  on  the  increased  non-radar  and  crossing  traffic  that  may  result  from  RVSM 
implementation. 

No  statistically  significant  differences  existed  between  scenarios  for  physical  environment 
ratings.  The  average  rating  reported  for  physical  environment  was  6.84  {SD  =  1.14).  The  most 
common  reasons  for  lower  ratings  were  differences  in  the  communication  system,  lights  were  too 
bright,  and  improper  location  of  the  A  position. 

Statistically  significant  differences  were  found  between  average  taskload  ratings  reported  for  the 
two  Areas,  F(4,  296)  =  2.42.  Average  ratings  of  6.24  through  6.50  {SD  =  1 .36)  were  reported  for 
Area  4  scenarios.  A  statistically  significant  lower  average  rating  of  5.66  {SD  =  1 .52)  was 
reported  for  Area  2.  The  most  commonly  reported  reason  for  a  lower  rating  was  the  lack  of 
weather  deviations.  Scenarios  for  Area  2  did  not  include  any  simulated  adverse  weather. 

3.  Results 

A  comparative  analysis  was  conducted  between  CVSM  and  RVSM  (e.g.,  1C  vs.  IR)  and 
between  RVSM  and  RVSM-E  (e.g.,  IR  vs.  IE).  CVSM  was  not  compared  to  RVSM-E  because 
it  could  not  be  concluded  if  differences  were  due  to  RVSM  or  contingency  operations. 

Controller  and  T/O  data  were  analyzed  independently  for  each  scenario.  For  both  controller  and 
T/O  data,  each  dependent  variable  was  analyzed  with  a  repeated  measures  design  using 
separation  as  the  within  subjects  variable  and  sector  and  position  as  the  between  subject 
variables.  All  tests  were  performed  with  a  significance  level  of  a  =  0.05.  Preliminary  analysis 
included  all  high-level  interactions.  If  the  interactions  did  not  reach  significance,  they  were 
discounted,  and  the  analysis  was  conducted  again  without  them.  It  should  be  noted  that  all  T/O 
data  are  expert  opinions  and  reflects  the  T/O’s  perception  of  the  controller’s  experiences.  The 
means  and  standard  deviations  of  all  the  subjective  and  objective  measures  are  given  in 
Appendix  D. 

Interval  workload  data  were  analyzed  with  number  of  aircraft  as  a  covariant.  The  choice  of  the 
number  of  aircraft  as  a  covariant  was  based  on  prior  literature,  indicating  a  strong  relationship 
between  workload  and  the  number  of  aircraft  (Costa,  1993;  Hurst  &  Rose,  1978  a,  b; 

Kopardekar,  1995;  Laurig,  Becker-Biskabom,  &  Reiche,  1971;  Stein,  1985;  Zeier,  1994). 

If  a  variable  was  found  to  be  statistically  significant,  its  practical  significance  was  assessed.  For 
purposes  of  this  analysis,  the  8-point  scale  was  segregated  into  three  levels:  low,  moderate,  and 
high.  If  the  variable  averages  remained  within  the  low  end  of  the  scale,  the  variable  was 
considered  not  affected  by  the  new  condition  at  a  level  that  had  any  real  world  application.  In 
other  words,  if  a  statistically  significant  variable  showed  a  change  at  the  moderate  or  high  level, 
it  was  considered  practically  significant.  A  workload  result  was  considered  practically 
significant  if  at  least  one  average  rating  was  above  2.99.  All  other  subjective  measures  were 
practically  significant  if  at  least  one  average  rating  was  less  than  6.01 . 


19 


The  first  section  discusses  operational  errors  and  deviations  recorded  during  the  simulation. 

Next,  a  comparative  analysis  of  CVSM  vs.  RVSM,  then  RVSM  vs.  RVSM-E  is  presented.  Each 
Post-Run  Questionnaire  parameter  was  analyzed  separately.  In  the  final  section,  the  effects  of 
increased  traffic  and  safety  issues  are  discussed. 

3.1  Operational  Errors 

Figures  10,  11,  12,  and  1 3  display  the  number  of  deviations  and  operational  errors  recorded  by 
the  T/Os  for  each  scenario.  Symbols  represent  the  total  number  of  operational  errors/deviations 
recorded  per  run.  Solid  squares  represent  operational  errors,  deviations  are  solid  triangles,  solid 
diamonds  indicate  an  equal  number  of  operational  errors  and  deviations,  and  open  squares 
indicate  that  no  operational  errors  or  deviations  were  recorded.  Refer  to  Appendix  E  for  a 
detailed  description  of  the  data  presented  in  the  figures. 

Figure  10  displays  results  from  Scenario  1.  As  shown  in  the  figure,  two  deviations  were  made  in 
sector  R60  during  CVSM.  These  were  made  by  the  same  control  team  and  were  related  to  flight 
strip  management.  One  operational  error  was  recorded  for  sector  R63,  and  it  related  to  lateral 
separation.  No  operational  errors  or  deviations  were  recorded  during  RVSM.  During  RVSM-E 
conditions,  two  operational  errors  were  made  by  the  same  team  pertaining  to  lateral  separation. 

A  different  team  also  had  a  lateral  separation  error.  Another  team  made  one  deviation  related  to 
flight  strips. 

Figure  1 1  shows  that  no  operational  errors  or  deviations  were  made  during  CVSM  and  RVSM 
for  Scenario  2.  During  RVSM-E,  two  teams  each  made  one  operational  error  related  to  lateral 
separation  in  sector  R60.  No  operational  errors  or  deviations  were  recorded  for  sector  R62/63 
during  RVSM-E. 

Figure  12  reveals  the  operational  errors  and  deviations  made  during  Scenario  3.  The  same 
control  team  made  two  deviations  in  sector  R60/62/63  during  CVSM.  One  operational  error 
related  to  lateral  separation  was  also  made  by  a  different  team.  No  operational  errors  or 
deviations  were  recorded  for  sector  R1  during  CVSM.  The  same  control  team  that  had  the 
deviations  during  CVSM  made  one  deviation  during  RVSM.  In  sector  Rl,  a  different  control 
team  made  two  operational  errors  related  to  separation.  Another  team  made  an  operational  error 
related  to  coordination  and  separation  for  sector  Rl. 

Figure  13  displays  results  from  Scenario  4.  The  figure  shows  that  during  CVSM,  one  team  made 
three  operational  errors  and  two  other  teams  made  one  operational  error  each.  All  errors  were 
related  to  separation.  During  RVSM,  one  team  made  two  errors.  One  error  was  an  RVSM 
violation  and  the  other  was  related  to  separation.  Two  different  teams  made  one  operational 
error  and  deviation  each.  For  both  teams,  the  operational  error  related  to  separation  and  the 
deviation  related  to  a  failure  to  coordinate.  Another  team  had  one  deviation  and  three  operational 
errors.  Two  of  the  errors  were  RVSM  violations  and  the  third  was  related  to  separation.  The 
deviation  was  related  to  a  failure  to  coordinate.  Figure  13  indicates  that  during  RVSM-E,  the 
same  team  made  five  operational  errors  and  two  deviations.  Both  deviations  were  related  to  a 
failure  to  coordinate.  All  of  the  errors  were  related  to  separation  and,  specifically,  one  error 
resulted  from  a  controller  not  following  the  command  that  was  coordinated.  Another  team,  had 
two  RVSM  violations  that  are  represented  here  as  operational  errors.  A  different  team  had  at 
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Run#  12  3  4  5  12  3  4  5  1  2  3  4  5  12  3  4  5  1  2  3  12  3 

R60  R63  R60  R63  R60  R63 

CVSM  RVSM  RVSM-E 


Separation  Type 

Figure  10.  Number  of  operational  errors  and  deviations  for  Scenario  1. 


Run#  12  3  4  5  12  3  4  5  1  2  3  4  5  12  3  4  5  1  2  1  2 

R60  R62/63  R60  R62/63  R60  R62/63 

CVSM  RVSM  RVSM-E 

Separation  Type 

Figure  1 1.  Number  of  operational  errors  and  deviations  for  Scenario  2. 
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LEGEND 


Run#  1  2  3  4  5  1  2  3  4  5  12  3  4 

R62/63  R62/63  R62/63 

CVSM  RVSM  RVSM-E 

Separation  Type 

Figure  13.  Number  of  operational  errors  and  deviations  for  Scenario  4. 
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least  three  errors  related  to  separation.  The  T/0  indicated  that,  for  this  team,  there  were  too  many 
errors  to  note  them  all.  Another  team  also  had  three  operational  errors.  Two  were  related  to 
separation  and  the  third  was  related  to  a  failure  to  coordinate. 

3.2  CVSM  vs.  RVSM 

3.2.1  Maintaining  Separation 

Analysis  of  controller  and  T/0  ratings  for  ability  to  maintain  aircraft  separation  revealed  no 
significant  difference  between  CVSM  and  RVSM  for  Scenarios  1,  2,  and  4.  The  result  failed  to 
indicate  any  impact  on  maintaining  separation  due  to  RVSM. 

Controller  self-ratings  revealed  a  significant  separation  by  sector  interaction  for  Scenario  3 
[F(l,  16)  =  9.02].  An  analysis  of  self-ratings  by  sector  resulted  in  a  significant  difference 
between  RVSM  and  CVSM  for  sector  R1  [F(l,  8)  =  10.29].  The  average  rating  was  lower  for 
RVSM  than  CVSM.  This  was  confirmed  by  T/0  ratings,  which  also  had  a  significant  separation 
by  sector  interaction  for  Scenario  3  [F(l,  18)  =  4.42].  T/O-rated  averages  also  remained 
relatively  constant  for  sector  R60/62/63  but  decreased  under  RVSM  for  sector  Rl. 

3.2.2  Workload 


Controllers  and  T/Os  gave  workload  ratings  at  four  intervals  throughout  each  run.  To 
supplement  these  ratings,  a  post-run  workload  rating  was  given  at  the  end  of  the  run.  Both  of 
these  ratings  are  discussed  for  each  scenario  below. 

Scenario  1  self-rated  interval  workload  was  significantly  different  between  CVSM  and  RVSM 
[F(l,  75)  =  6.26].  The  average  workload  was  lower  during  CVSM  than  RVSM.  T/0  ratings 
showed  a  significant  sector  by  separation  interaction  for  Scenario  1  [F(l,  75)  =  6.26].  An 
analysis  by  sector  confirmed  the  controller  finding  only  for  sector  R63  [F(l ,  37)  =  9. 10].  All 
averages  fell  below  2.99,  therefore,  the  significance  found  for  the  controller  and  T/O  ratings  was 
considered  not  to  be  practical. 

Analysis  showed  a  significant  difference  in  the  self-rated  post-run  workload  between  CVSM  and 
RVSM  for  Scenario  1  [F(l,  16)  =  12.23].  Both  controller  averages  were  below  2.99,  therefore, 
the  significance  was  considered  not  practical.  T/O  ratings  showed  a  significant  separation  by 
sector  interaction  for  Scenario  1  [F(l,  16)  =  7.38].  An  analysis  by  sector  showed  a  significant 
difference  in  the  ratings  for  sector  R63  [F(l,  8)  =  19.60].  The  RVSM  average  rating  for  sector 
R63  was  higher  than  the  CVSM  average. 

Scenario  2  self-rated  interval  workload  was  significantly  different  between  CVSM  and  RVSM 
[F(l,  55)  =  4.98].  Both  controller  averages  were  below  2.99,  therefore,  the  significance  was 
considered  not  practical.  T/O  interval  ratings  showed  no  significant  difference  between  CVSM 
and  RVSM.  Both  measures  failed  to  indicate  any  practical  impact  on  interval  workload  for 
Scenario  2  due  to  RVSM. 
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Both  self-  and  T/O-rated  post-run  workload  analyses  resulted  in  no  significant  difference 
between  CVSM  and  RVSM  for  Scenario  2.  This  result  failed  to  indicate  any  impact  on  post-run 
workload  due  to  RVSM. 

Both  self-  and  T/O-rated  interval  workload  analyses  resulted  in  no  significant  difference  between 
CVSM  and  RVSM  for  Scenario  3.  This  result  failed  to  indicate  any  impact  on  interval  workload 
due  to  RVSM. 

Post-run  analysis  was  not  consistent  with  interval  workload  analysis.  Scenario  3  self-rated  post¬ 
run  workload  had  a  significant  separation  by  sector  interaction  [F(l,  16)  =  1 1.20].  An  analysis 
by  sector  showed  that  sector  R1  workload  was  significantly  different  between  CVSM  and  RVSM 
[F(l,  8)  =  9.80].  The  RVSM  average  rating  was  higher  than  the  CVSM  average.  This  was 
confirmed  by  T/0  ratings  which  also  revealed  a  separation  by  sector  interaction  for  Scenario  3 
[F(l,  16)  =  5.69].  Analysis  by  sector  also  showed  a  significant  difference  in  the  T/O-rated 
workload  for  sector  R1  [(F(l,  8)  =  7.58].  Here  again,  the  RVSM  average  rating  was  higher  than 
the  CVSM  average. 

Scenario  4  self-rated  interval  workload  was  significantly  different  between  CVSM  and  RVSM 
[F(l,  58)  =  1 1.29].  T/O  ratings  also  showed  a  significant  difference  between  CVSM  and  RVSM 
[F(l,  58)  =  1 1.68].  The  average  interval  workload  for  both  the  controllers  and  T/Os  was  lower 
for  CVSM  than  RVSM. 

The  average  self-rated  post-run  workload  was  significantly  different  for  Scenario  4  [F(l,  12)  = 
5.76].  The  RVSM  average  rating  was  higher  than  the  CVSM  average.  Although  not  significant, 
the  average  T/O  workload  rating  for  RVSM  was  also  higher  than  the  average  T/O  rating  for 
CVSM. 

3.2.3  Frequency  of  Communications 

For  all  scenarios,  no  significant  difference  was  found  between  CVSM  and  RVSM  for  the  average 
recorded  number  of  communications.  Results  failed  to  indicate  any  practical  impact  on  the 
frequency  of  controller  communications  due  to  RVSM. 

3.2.4  Communication  Rating 

Self-  and  T/O-rated  values  for  communication  were  analyzed  and  no  statistically  significant 
difference  was  found  between  CVSM  and  RVSM  for  Scenarios  1 , 2,  and  4.  These  results  failed 
to  indicate  any  impact  on  controller’s  ability  to  follow  ATP  communication  specifications  due  to 
RVSM. 

Scenario  3  self-ratings  showed  a  significant  separation  by  sector  interaction  [F(l,  16)  =  6.45]. 
Analysis  by  sector  revealed  a  significant  difference  between  RVSM  and  CVSM  for  sector  R1 
[F(l,  9)  =  4.97].  Both  controller  average  ratings  were  above  6.01,  therefore,  the  significance  was 
considered  not  practical.  T/O  ratings  also  showed  a  significant  separation  by  sector  interaction 
for  Scenario  3  [F(l,  16)  =  5.24].  Analysis  by  sector  confirmed  a  significant  RVSM  effect  for 
sector  R1  [F(l,  8)  =  5.40]. 
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3.2.5  Coordination  Rating 


Analysis  of  T/0-  and  self-rated  values  for  coordination  showed  no  statistically  significant 
difference  between  CVSM  and  RVSM  for  Scenarios  1,  2,  and  4.  These  results  failed  to  indicate 
any  impact  on  controller  coordination  due  to  RVSM. 

Analysis  of  self-ratings  showed  that  Scenario  3  had  a  significant  separation  by  position 
interaction  [F(l,  16)  =  8.38].  An  analysis  by  position  showed  a  significant  separation  effect  for 
the  R  position  [F(l,  8)  =  6.45].  The  average  R  position  rating  decreased  from  CVSM  to  RVSM. 
Both  controller  average  ratings  were  above  6.01,  therefore,  the  significance  was  considered  not 
practical.  T/0  ratings  resulted  in  no  significant  difference  between  CVSM  and  RVSM  for 
Scenario  3.  Both  controller  and  T/O  measures  failed  to  indicate  any  practical  impact  on 
controller  coordination  due  to  RVSM  for  Scenario  3. 

3.2.6  Situational  Awareness 


Analysis  of  controller  and  T/O  ratings  for  situational  awareness  showed  no  statistically 
significant  difference  between  CVSM  and  RVSM  for  Scenarios  1,  2,  and  4.  These  results  failed 
to  indicate  any  impact  on  controller  situational  awareness  due  to  RVSM. 

Scenario  3  self-rated  analysis  revealed  a  significant  separation  by  sector  interaction 

[F(l,  16)  =  6.03].  The  average  rating  decreased  from  CVSM  to  RVSM  for  sector  Rl.  Although 

not  significant,  the  same  trend  was  observed  for  T/O  ratings. 

3.2.7  Priority  of  Actions 

Self-  and  T/O-rated  values  for  priority  of  actions  were  analyzed  and  revealed  no  significant 
difference  between  CVSM  and  RVSM  for  Scenarios  2,  3,  and  4.  These  results  failed  to  indicate 
any  impact  on  the  controller’s  ability  to  issue  instructions  in  a  prioritized  manner  due  to  RVSM. 

Self-rated  values  for  Scenario  1  showed  a  significant  separation  by  position  interaction 
[F(l,  16)  =  5.12].  Analysis  by  control  position  revealed  a  significant  separation  effect  for  the  R 
position  only  [F(l,  7)  =  5.89].  Both  controller  average  ratings  were  above  6.01,  therefore,  the 
significance  was  considered  not  practical.  T/O  ratings  indicated  no  significant  difference 
between  CVSM  and  RVSM  for  Scenario  1.  Both  controller  and  T/O  measures  failed  to  indicate 
any  practical  impact  on  priority  of  actions  for  Scenario  1  due  to  RVSM. 

3.2.8  Flight  Strips 

Analysis  of  self-  and  T/O-rated  values  for  flight  strips  showed  no  significant  difference  between 
CVSM  and  RVSM  for  Scenarios  2,  3,  and  4.  These  results  failed  to  indicate  any  impact  on  the 
controller’s  ability  to  manage  flight  strips  due  to  RVSM. 

Controller  self-ratings  had  a  significant  separation  by  position  interaction  for  Scenario  1 
[F(l,  16)  =  5.94].  An  analysis  by  control  position  showed  that  only  the  R  control  position  had  a 
significant  separation  effect  [F(l,  8)  =  8.26].  The  average  decreased  from  CVSM  to  RVSM  for 
the  R  controller.  T/O-rated  values  did  not  confirm  this  interaction.  Instead  a  significant 
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separation  by  sector  interaction  was  found  for  Scenario  1  [F(l,  16)  =  5.95].  Analysis  by  sector 
showed  that  sector  R63  had  significantly  different  flight  strip  ratings  between  CVSM  than 
RVSM  [F(l ,  8)  =  8.79].  The  average  rating  for  CVSM  was  higher  than  the  average  for  RVSM. 

3.2.9  Computer  Entry 

Analysis  of  self-rated  and  T/O-rated  values  for  computer  entry  resulted  in  no  significant 
difference  between  CVSM  and  RVSM  for  Scenarios  2,  3,  and  4.  These  results  failed  to  indicate 
any  impact  on  the  controller’s  ability  to  accurately  input  control  commands  due  to  RVSM. 

T/O-rated  values  for  Scenario  1  showed  a  significant  separation  by  sector  interaction 
[F(l,  16)  =  10.08].  An  analysis  by  sector  revealed  that  only  sector  R60  had  significantly 
different  averages  between  CVSM  and  RVSM  [F(l,  8)  =  6.74].  Both  T/0  average  ratings  were 
above  6.01,  therefore,  the  significance  was  considered  not  practical.  Controller  self-ratings 
indicated  no  significant  difference  between  CVSM  and  RVSM  for  Scenario  1.  Both  controller 
and  T/0  measures  failed  to  indicate  any  practical  impact  on  the  controller’s  ability  to  accurately 
input  control  commands  due  to  RVSM  for  Scenario  1. 

3.2.10  SAR  Variables 


Analysis  of  various  SAR  variables  showed  no  significant  difference  between  CVSM  and  RVSM 
for  ACCPT,  AM,  ERROR,  QF  QP,  QQ,  QR,  QX,  REJECT,  or  SR  messages  for  all  scenarios. 
Refer  to  Table  5  for  a  complete  list  of  SAR  acronyms  and  their  meanings. 

Analysis  of  SAR  recordings  for  Scenario  1  showed  a  significant  difference  between  CVSM  and 
RVSM  for  the  number  of  QN  messages  [F(l,  17)  =  5.37].  The  average  number  of  QN  messages 
was  lower  during  RVSM  than  CVSM.  A  significant  separation  by  position  effect  was  found  for 
QZ  messages  [F(l,  17)  =  4.27].  An  analysis  by  position  showed  that  only  the  R  control  position 
had  a  significant  difference  [F(l,  8)  =  9.01].  The  average  number  of  QZ  messages  for  the  R 
controller  was  greater  for  RVSM  than  CVSM. 

The  number  of  QZ  messages  was  significantly  different  between  CVSM  and  RVSM  for  Scenario 
2  [Ffl,  9)  =  7.15].  The  average  number  of  messages  was  greater  during  RVSM  than  CVSM. 

Analysis  of  SAR  recordings  for  Scenario  4  showed  a  significant  difference  between  CVSM  and 
RVSM  for  QT  messages  [F(l,  4)  =  7.84],  QU  messages  [F(l,  5)  =  7.50],  and  QZ  messages  [F(l, 
4)  =  12.60].  The  average  number  of  messages  was  greater  during  RVSM  than  CVSM  for  all 
three  message  types. 

3.3  RVSM  vs.  RVSM-E 

Scenario  3  did  not  have  any  runs  with  contingency  operations,  therefore,  it  is  not  included  in  this 
section. 
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3.3.1  Maintaining  Separation 


Analysis  of  controller  self-ratings  for  maintaining  separation  showed  a  significant  difference 
between  RVSM  and  RVSM-E  for  Scenario  1  [F(l,  8)  =  28.80].  The  average  rating  was  lower  for 
RVSM-E  than  RVSM.  T/0  ratings  had  a  significant  sector  by  separation  interaction  for 
Scenario  1  [F(l,  8)  =  6.88].  An  analysis  by  sector  revealed  a  significant  separation  effect  for  R60 
[F(l,  4)  =  7.60].  The  average  rating  for  RVSM-E  was  less  than  the  average  for  RVSM. 

Analysis  of  self-rated  values  for  maintaining  separation  showed  a  significant  difference  between 
RVSM  and  RVSM-E  for  Scenario  2  [F(l,  4)  =  25.00].  Both  controller  average  ratings  were 
above  6.01,  therefore,  the  significance  was  considered  not  practical.  An  analysis  of  T/O  ratings 
also  showed  a  significant  difference  between  RVSM  and  RVSM-E  for  Scenario  2 
[F(l,  4)  =  7.70].  The  average  for  RVSM-E  was  lower  than  the  average  for  RVSM. 

Analysis  of  self-rated  values  for  separation  showed  a  significant  difference  between  RVSM-E 
and  RVSM  for  Scenario  4  [F(l,  7)  =  9.39].  An  analysis  of  T/O  ratings  showed  the  same 
[F(l,  7)  =  9.58].  For  Scenario  4,  the  average  ratings  for  separation  were  lower  during  RVSM-E 
than  RVSM  for  both  controllers  and  T/Os. 

3.3.2  Workload 


The  following  section  discusses  interval  and  post-run  workload  ratings.  The  average  T/O  and 
controller  workload  ratings  were  higher  for  RVSM-E  than  RVSM.  All  average  ratings  follow  the 
same  trend  regardless  of  their  statistical  significance  or  who  gave  the  rating. 

The  self-rated  interval  workload  was  significantly  different  between  RVSM  and  RVSM-E  for 
Scenario  1  [(F(l,  43)  =  8.23].  An  analysis  of  T/O  interval  ratings  showed  the  same 
[F(l,  43)  =  9.52].  During  Scenario  1,  the  average  interval  workload  was  higher  for  RVSM-E 
than  RVSM  for  both  controllers  and  T/Os. 

An  analysis  of  Scenario  1  post-run  workload  further  supported  the  workload  difference  between 
RVSM  and  RVSM-E.  The  self-rated  post-run  workload  was  significantly  different  between 
RVSM  and  RVSM-E  [F(l,  8)  =  10.26].  An  analysis  of  T/O  post-run  workload  showed  the  same 
[F(l,  8)  =  43.68].  During  Scenario  1,  the  average  post-run  workload  was  higher  for  RVSM-E 
than  RVSM  for  both  controllers  and  T/Os. 

Analysis  of  self-  and  T/O-rated  values  for  interval  and  post-run  workload  resulted  in  no 
significant  difference  between  RVSM  and  RVSM-E  for  Scenario  2.  The  controller  and  T/O 
ratings  failed  to  indicate  any  impact  on  workload  for  Scenario  2  due  to  contingency  operations. 

A  significant  difference  between  RVSM  and  RVSM-E  was  found  for  T/O  rated  interval 
workload  for  Scenario  4  [F(l,  44)  =  19.94].  The  average  interval  workload  was  higher  for 
RVSM-E  than  RVSM.  This  was  not  confirmed  by  self-ratings,  which  resulted  in  no  significant 
difference  between  CVSM  and  RVSM  for  Scenario  4. 

An  analysis  of  T/O  post-run  workload  further  supported  the  workload  difference  between  RVSM 
and  RVSM-E  for  Scenario  4.  The  T/O-rated  post-run  workload  was  significantly  different 
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between  RVSM  and  RVSM-E  [F(l,  9)  =  13.02],  No  separation  effect  was  found  for  controller 
ratings.  During  Scenario  4,  the  average  post-run  workload  was  higher  for  RVSM-E  than  RVSM 
for  both  controllers  and  T/Os. 

3.3.3  Frequency  of  Communications 

Analysis  of  the  number  of  recorded  communications  revealed  a  significant  difference  in  the 
number  of  communications  between  RVSM  and  RVSM-E  for  Scenario  1  [F(l,  8)  =  64.83]  and 
Scenario  2  [F(l,  4)  =  6.63].  For  both  scenarios,  the  average  frequency  of  communications  was 
higher  for  RVSM-E  than  RVSM. 

Scenario  4  had  a  significant  separation  by  position  interaction  [F(2,  9)  =  10.05].  The  R  position 
experienced  a  22%  increase  in  the  average  number  of  communications  under  RVSM-E  compared 
to  RVSM.  The  A  position  average  more  than  doubled.  However,  the  D  position  experienced  a 
27%  decrease  in  the  average  number  of  communications  during  RVSM-E  compared  to  RVSM. 

3.3.4  Communication  Rating 

No  significant  difference  between  RVSM  and  RVSM-E  was  found  for  Scenarios  2  and  4  for  both 
self-  and  T/O-ratings.  These  results  failed  to  indicate  any  impact  on  the  controller’s  ability  to 
follow  ATP  communication  specifications  due  to  contingency  operations. 

An  analysis  of  self-ratings  for  communication  showed  that  Scenario  1  had  a  significantly 
different  average  rating  for  RVSM  than  RVSM-E  [F(l,  8)  =  24.00].  The  average  rating  for 
RVSM  was  higher  than  the  average  for  RVSM-E.  This  was  confirmed  by  Scenario  1  T/0 
ratings,  which  also  had  significantly  different  ratings  [F(l,  11)=  12.75].  The  average  T/0  rating 
given  during  RVSM  was  also  higher  than  the  average  for  RVSM-E.  However,  all  average  ratings 
were  above  6.01,  therefore,  the  significance  found  for  the  controller  and  T/O  ratings  was 
considered  not  practical. 

3.3.5  Coordination  Rating 

Analysis  of  self-ratings  for  coordination  showed  that  Scenario  1  had  a  significant  difference 
between  RVSM  and  RVSM-E  [F(l,  8)  =  5.35].  The  average  rating  for  RVSM  was  higher  than 
the  average  for  RVSM-E.  An  analysis  of  T/O  ratings  confirmed  that  Scenario  1  had  significantly 
different  ratings  [F(l,  8)  =  17.29].  Both  of  these  average  ratings  were  above  6.01,  therefore,  the 
significance  of  the  T/O  ratings  was  considered  not  practical. 

Analysis  of  self-ratings  for  coordination  showed  that  Scenario  2  had  a  significantly  different 
average  value  for  RVSM  than  RVSM-E  [F(l,  4)  =  9.00].  Both  controller  average  ratings  were 
above  6.01,  therefore,  the  significance  was  considered  not  practical.  T/O  ratings  indicated  no 
significant  difference  between  RVSM  and  RVSM-E  for  Scenario  2.  Both  controller  and  T/O 
measures  failed  to  indicate  any  practical  impact  on  controller  coordination  for  Scenario  2  due  to 
contingency  operations. 
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Scenario  4  resulted  in  no  significant  difference  between  RVSM  and  RVSM-E  for  both  T/0-  and 
self-ratings.  The  controller  and  T/0  ratings  failed  to  indicate  any  impact  on  controller 
coordination  for  Scenario  4  due  to  contingency  operations. 

3.3.6  Situational  Awareness 


No  significant  difference  between  RVSM  and  RVSM-E  was  found  for  Scenarios  2  and  4  for  both 
self-  and  T/O-ratings.  These  results  failed  to  indicate  any  impact  on  the  controller’s  situational 
awareness  due  to  contingency  operations.  Although  not  significant,  the  averages  for  RVSM-E 
were  lower  than  RVSM. 

The  self-rated  values  for  situational  awareness  were  significantly  different  between  RVSM-E  and 
RVSM  for  Scenario  1  [F(l,  8)  =  60.75].  The  average  rating  was  lower  for  RVSM-E  than 
RVSM.  This  was  confirmed  by  T/0  ratings  only  for  sector  R60.  T/0  ratings  had  a  significant 
separation  by  sector  interaction  [F(l,  8)  =  7.84].  An  analysis  by  sector  showed  that  sector  R60 
had  a  significant  difference  between  RVSM-E  and  RVSM  [F(l,  4)  =  15.43].  The  average  T/0 
situational  awareness  rating  for  sector  R60  was  lower  during  RVSM-E  than  RVSM. 

3.3.7  Priority  of  Actions 

Analysis  of  self-rated  values  for  priority  of  actions  showed  a  significant  difference  between 
RVSM  and  RVSM-E  for  Scenario  1  [F(l,  8)  =  8.07].  T/O  ratings  also  showed  a  significant 
difference  [F(l,  8)  =  12.00].  The  average  rating  for  priority  of  actions  was  lower  during  RVSM- 
E  than  RVSM  for  both  controllers  and  T/Os.  All  average  ratings  were  above  6.01,  therefore,  the 
significance  found  for  the  controller  and  T/O  ratings  was  considered  not  practical. 

T/O  ratings  for  Scenario  2  showed  a  significant  difference  between  RVSM  and  RVSM-E 
[F(l,  4)  =  6.58].  Both  T/O  average  ratings  were  above  6.01,  therefore,  the  significance  was 
considered  not  practical.  Controller  self-ratings  indicated  no  significant  difference  between 
RVSM  and  RVSM-E  for  Scenario  2.  Both  controller  and  T/O  measures  failed  to  indicate  any 
practical  impact  on  the  controller’s  priority  of  actions  for  Scenario  2  due  to  contingency 
operations. 

T/O  ratings  also  showed  a  significant  difference  for  Scenario  4  [F(l,  8)  =  5.70].  The  average 
rating  for  RVSM-E  was  lower  than  RVSM.  Although  the  difference  was  not  statistically 
different,  self-rated  averages  were  also  lower  for  RVSM-E  than  RVSM  for  Scenario  4. 

3.3.8  Flight  Strips 

No  significant  difference  was  found  for  Scenarios  2  and  4  for  both  self-  and  T/O-ratings.  These 
results  failed  to  indicate  any  impact  on  the  controller’s  ability  to  manage  flight  strips  due  to 
contingency  operations.  Although  not  significant,  the  averages  for  RVSM-E  were  lower  than 
RVSM. 

Scenario  1  self-rated  values  for  flight  strips  were  significantly  different  between  RVSM-E  and 
RVSM  [F(l,  8)  =  7.04].  The  average  self-rating  was  lower  for  RVSM-E  than  RVSM.  T/O 
ratings  showed  a  significant  sector  by  separation  interaction  for  Scenario  1  [F(l,  8)  =  9.63].  An 
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analysis  by  sector  showed  that  only  sector  R60  had  a  significant  separation  effect 

[F(l,  8)  =  8.79].  The  average  T/0  rating  for  sector  R60  was  lower  for  RVSM-E  than  RVSM. 

3.3.9  Computer  Entry 

Self-reported  ratings  for  computer  entry  were  significantly  different  between  RVSM  and  RVSM- 
E  for  Scenario  1  [F(l,  9)  =  5.14].  The  average  self-rating  was  lower  for  RVSM-E  than  RVSM 
[F(l,  8)  =  9.00].  An  analysis  by  sector  showed  that  sector  R60  had  a  significant  separation 
effect.  However,  all  average  ratings  were  above  6.01,  therefore,  the  significance  found  for  the 
controller  and  T/O  ratings  was  considered  not  practical. 

Scenario  2  average  self-ratings  for  computer  entry  had  a  significant  separation  by  position 
interaction  [F(l,  4)  =  9.00].  The  average  value  was  lower  for  the  R  position  during  RVSM-E  as 
compared  to  RVSM.  Both  controller  average  ratings  were  above  6.01,  therefore,  the  significance 
was  considered  not  practical.  T/O  ratings  indicated  no  significant  difference  between  RVSM  and 
RVSM-E  for  Scenario  2.  Both  T/O  and  controller  measures  failed  to  indicate  any  practical 
impact  on  the  controller’s  ability  to  accurately  input  control  commands  for  Scenario  2  due  to 
contingency  operations. 

Scenario  4  controller  ratings  revealed  a  significant  separation  by  position  interaction 

[F(2,  7)  =  7.83].  An  analysis  by  position  showed  that  the  R  position  had  a  significant  separation 

effect  [F(l ,  3)  =  49.00].  The  average  R  position  rating  was  lower  for  RVSM-E  than  RVSM. 

T/O  ratings  for  Scenario  4  also  revealed  a  significant  separation  by  position  interaction 
[F(l,  8)  =  8.91].  An  analysis  of  T/O  ratings  by  position  also  showed  that  the  R  position  had  a 
separation  effect  [F(l,  3)  =  16.33].  The  average  T/O  rating  for  the  R  position  was  also  lower  for 
RVSM-E  than  RVSM. 

3.3.10  SAR  Variables 

Analysis  of  SAR  variables  resulted  in  no  significant  difference  between  RVSM  and  RVSM-E  for 
ACCPT,  AM,  ERROR,  QF,  QN,  QP,  QQ,  QR,  QU,  QZ,  or  SR  messages.  Scenario  2  is  not 
analyzed  here  because  not  enough  runs  were  recorded.  Refer  to  Table  5  for  a  complete  list  of 
SAR  acronyms  and  their  meanings. 

Analysis  of  QT  messages  showed  that  Scenario  1  had  a  significant  separation  by  position 
interaction  [F(l,  9)  =  30.07].  When  values  were  analyzed  separately  by  position,  the  R  controller 
showed  a  ignificant  separation  effect  [F(l ,  4)  =  36.75].  The  average  number  of  recorded 
messages  decreased  for  the  R  controller  from  RVSM  to  RVSM-E. 

Analysis  of  QT  messages  showed  that  Scenario  4  had  a  significant  separation  by  position 
interaction  [F(  1 , 4)  =  9. 1 5] .  The  average  number  of  QT  messages  decreased  for  the  R  controller 
from  RVSM  to  RVSM-E.  Analysis  of  QX  messages  showed  a  significant  difference  between 
RVSM-E  and  RVSM  [F(l,  5)  =  7.50].  During  Scenario  4,  the  average  number  of  QX  messages 
was  lower  for  RVSM-E  than  RVSM. 
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3.4  Safety  Issues 


There  were  173  answers  to  question  4  of  the  Post-Run  Questionnaire,  “What  was  your  primary 
safety  concern  during  the  last  simulation  run?”  Of  the  173  answers,  30  answers  were  not 
analyzed  as  safety  concerns  directly  related  to  the  procedure.  Of  the  30  discounted  answers,  12 
described  experimental  parameters  as  a  concern  (i.e.,  controllers  did  not  fully  understand  the 
RVSM  rules),  13  answers  indicated  the  fidelity  of  the  simulation  (i.e.,  inefficient  strip  printing 
during  start  up),  and  5  answers  merely  stated  that  the  participant  had  no  safety  concerns.  Of  the 
remaining  143  answers,  150  distinct  responses  were  identified. 

The  safety  concerns  were  categorized  into  five  main  groups:  Separation,  Coordination,  Data 
Blocks,  Transitioning,  and  Other.  Since  the  data  were  categorical,  statistical  chi-square  analyses 
were  completed  where  appropriate.  Figure  14  characterizes  the  overall  proportion  of  each  type  of 
safety  concern  by  separation  condition.  As  shown  in  the  figure,  the  primary  safety  concern  was 
maintaining  separation  regardless  of  the  separation  minima.  During  contingency  operations, 
maintaining  separation  took  even  higher  priority. 

A  chi-square  analysis  did  not  result  in  significant  differences  between  CVSM  and  RVSM  safety 
concerns.  Transitioning  was  not  a  concern  during  CVSM  runs  because  it  is  logically  not 
applicable.  RVSM-E  was  examined  separately  because  safety  concerns  were  logically  expected 
to  be  substantially  different  during  radar  outages  and  severe  weather.  Area  2  and  Area  4  safety 
concerns  were  also  tested  for  differences  because  of  the  differences  between  the  Areas. 

However,  their  differences  were  non-significant. 
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Figure  14.  Safety  concerns. 

3.5  Changes  in  Control  Strategies.  Procedures,  and  Equipment 

Questions  5  and  6  of  the  Post-Run  Questionnaire  (refer  to  Appendix  B)  inquired  about  changes 
in  control  strategies,  procedures,  or  equipment  due  to  RVSM.  Fifty-five  T/0  responses  and  104 
controller  responses  were  given  for  both.  For  question  5, 26%  of  the  104  controller  responses 
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were,  “yes,”  they  did  change  their  usual  control  strategies.  A  higher  percentage  of  T/Os’ 
responses,  38%,  answered,  “yes,”  the  controllers  changed  their  usual  control  strategies.  The 
response  to  question  6  was  similar  to  question  5.  Twenty-two  percent  of  the  controller  responses 
were,  “yes”  they  felt  some  equipment  would  have  to  be  changed  or  new  procedures  implemented 
while  36%  of  the  T/0  responses  were  “yes.” 

For  both  questions,  the  changes  included  giving  more  attention  to  altitude  assignments  and 
becoming  more  aware  of  boundaries  and  non-radar  traffic.  Of  those  that  responded  “no,”  most 
indicated  that  further  exposure  to  RVSM  would  increase  their  comfort  level  and  that  no 
additional  procedures  or  equipment  would  be  necessary. 

4.  Discussion 


When  interpreting  and  discussing  results,  the  reader  should  consider  that  sectors  Rl,  R60,  and 
R63  were  used  as  RVSM  transition  sectors.  This  was  not  a  study  of  full  RVSM  implementation 
in  the  WATRS  Area.  Findings  for  a  particular  sector  during  this  study  may  not  necessarily  be  the 
same  if  full  implementation  was  studied.  Additional  studies  may  need  to  be  conducted  to 
understand  the  impact  of  full  RVSM  implementation  within  the  WATRS  Area. 

4.1  CVSMvs.RVSM 


Results  from  Scenarios  1,  2,  and  3  did  not  indicate  a  clear  difference  between  CVSM  and  RVSM 
for  sectors  R60  and  R63  or  the  combined  sectors  R62/63  or  R60/62/63.  There  was  not  a 
significant  difference  between  CVSM  and  RVSM  for  most  subjective  or  objective  measures. 

The  recorded  number  of  operational  errors  did  not  increase  under  RVSM  for  Scenarios  1  and  2 
and  sector  R60/62/63  of  Scenario  3.  The  recorded  number  of  QZ  (assigned  altitude,  etc.) 
messages  increased  for  Scenarios  1  (R  position  only)  and  2  during  RVSM  conditions. 

Controllers  confirmed  that  this  was  due  to  assigning  more  altitudes  during  RVSM.  There  were 
significantly  less  QN  (data  block  offset,  force  data  bloek,  etc.)  messages  under  RVSM  than 
CVSM  for  Scenario  1.  The  lower  average  for  RVSM  was  thought  to  be  the  result  of  performing 
data  block  offset  and  force  data  block.  Because  these  two  funetions  are  more  cosmetic  than 
necessary  in  nature,  it  would  follow  that  controllers  would  be  less  likely  to  perform  these  actions 
when  they  have  other  higher  priority  actions  to  perform.  Controllers  also  mentioned  that  the  data 
blocks  were  almost  overlapping  in  some  cases.  However,  they  did  not  feel  RVSM  increased  data 
block  overlap  problems. 

Scenario  4  post-run  and  interval  workload  ratings  indicated  that  RVSM  was  significantly  more 
workload  intensive  that  CVSM  for  sector  R62/63.  Also,  a  greater  number  of  operational  errors 
and  deviations  were  recorded  during  RVSM  than  CVSM  for  Scenario  4.  Some  of  the  errors  were 
RVSM  violations.  This  seems  to  indicate  that  RVSM  transitions  have  a  greater  effect  on  the 
controller  when  they  are  working  with  a  higher  number  of  aircraft,  which  was  the  case  in 
Scenario  4.  In  addition,  a  greater  number  of  QT,  QU,  and  QZ  messages  were  recorded  during 
RVSM. 

As  previously  mentioned,  sector  Rl  was  the  only  sector  from  Area  2.  It  had  different  rules  for 
RVSM  transition  and  a  lot  more  crossing  traffic  than  the  other  sectors  from  Area  4.  Examination 
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of  all  the  results  clearly  indicate  that  for  sector  Rl,  studied  during  Scenario  3,  the  controllers 
incurred  a  greater  hardship  when  controlling  traffic  with  RVSM  than  with  CVSM.  Rl  had  lower 
ratings  during  RVSM  for  the  ability  to  maintain  separation,  communication,  and  situational 
awareness.  The  post-run  workload  was  higher  during  RVSM  than  CVSM.  In  support  of  the 
subjective  measures,  operational  errors  and  deviations  relating  to  separation  were  observed  for 
sector  Rl  during  RVSM,  but  no  operational  errors  or  deviations  were  observed  during  CVSM. 

4.2  RVSM  vs.  RVSM-E 

During  contingency  operations,  all  scenarios  showed  an  increase  in  the  frequency  of 
communications.  This  was  expected  because  a  contingency  event  usually  requires  additional 
communication  between  the  controller  and  pilot.  However,  there  was  no  indication  that  this 
increase  resulted  in  a  decreased  ability  to  follow  communication  specifications.  In  the  case  of 
three  controllers,  it  appears  that  the  A  controller  may  become  more  active  in  communications 
during  the  contingency  operation  thus  decreasing  the  number  of  communications  made  by  the  D 
controller  and  allowing  that  controller  to  focus  on  other  activities. 

Controllers  and  T/Os  agreed  that  RVSM-E  was  more  workload  intensive  than  RVSM  for 
Scenario  1.  Controllers  also  felt  that  they  had  more  difficulties  with  coordination,  situational 
awareness,  flight  strip  management,  and  maintaining  separation  during  RVSM-E.  T/Os  reported 
similar  results  for  sector  R60.  This  finding  was  supported  by  three  recorded  operational  errors 
and  deviations  for  sector  R60  during  RVSM-E,  which  were  related  to  maintaining  separation  and 
flight  strip  management.  Also,  it  was  found  that  regardless  of  the  sector,  the  R  controller  entered 
significantly  less  QT  (assigned  altitude,  coast  track,  track)  messages  during  the  RVSM-E 
condition.  This  is  most  likely  due  to  the  controller  becoming  more  focused  on  the  contingency 
events  and,  therefore,  not  assigning  as  many  altitudes. 

T/Os  felt  that  controllers  had  a  greater  difficulty  maintaining  separation  during  RVSM-E 
compared  to  RVSM  for  Scenario  2,  particularly  for  sector  R60.  This  is  supported  by  two 
operational  errors  related  to  separation  made  in  sector  R60  during  RVSM-E.  Neither  the 
controllers  nor  the  T/Os  felt  that  most  of  the  other  controller  capabilities  were  affected  by 
contingency  operations  during  this  scenario  (i.e.,  workload,  situational  awareness,  and  priority  of 
actions). 

T/Os  found  RVSM-E  more  workload  intensive  than  RVSM  for  Scenario  4.  Controller  post-run 
ratings  indicated  the  same.  The  ability  to  maintain  separation  and  priority  of  actions  were  also 
degraded  during  RVSM-E.  This  is  supported  by  the  numerous  operational  errors  and  deviations 
recorded  relating  to  separation  and  a  failure  to  perform  an  action.  There  was  also  indication  that 
the  contingency  operations  affected  the  R  controllers  computer  entry  abilities.  The  R  position 
had  significantly  less  QT  messages  during  RVSM-E  conditions.  Therefore,  during  contingency 
operations,  the  R  controller  could  not  assign  as  many  altitudes.  QX  (drop  tracks  only)  messages 
were  also  less  frequent  during  RVSM-E.  This  may  be  because  the  controllers  wanted  to  keep  an 
accurate  picture  of  all  the  aircraft  during  contingency  operations  or  they  were  too  busy  with 
higher  priority  actions. 
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4.3  Conflict  Alert 


The  participants  discussed  whether  it  is  better  to  have  the  Conflict  Alert  activate  too  many  times 
rather  than  going  off  too  few  times.  They  suggested  that  the  RVSM-equipped  aircraft  be 
identified  by  a  special  mark.  Therefore,  Conflict  Alert  for  these  aircraft  could  be  neglected  if  the 
alert  threshold  is  set  for  2000  ft.  They  also  indicated  that  it  might  be  possible  to  use  the 
suppressed  group  for  RVSM-equipped  aircraft;  thus,  the  Conflict  Alert  would  not  be  activated  at 
1000-ft  altitude  differential.  If  the  suppressed  group  arrangement  would  not  be  possible,  it  is 
preferred  that  the  Conflict  Alert  threshold  be  set  at  2000  ft. 

4.4  Impact  of  Bad  Weather  With  RVSM 

During  debrief  sessions,  participants  indicated  that  the  location  of  adverse  weather  would 
determine  the  impact  on  the  controllers  ability  to  control  traffic  during  RVSM.  They  mentioned 
that  if  the  bad  weather  were  closer  to  the  transition  boundary  with  RVSM,  it  would  considerably 
increase  their  workload.  However,  if  the  adverse  weather  were  not  close  to  the  transition 
boundary,  they  would  get  more  altitudes  and,  therefore,  have  more  flexibility  to  arrange  all 
aircraft  around  the  bad  weather. 

4.5  Increased  Traffic  Under  RVSM  Separation 

One  consideration  of  RVSM  implementation  is  whether  more  aircraft  can  be  managed  with 
RVSM  as  compared  to  CVSM.  To  address  this  issue,  this  section  discusses  workload  and 
operational  errors/deviations  for  sector  R62/63.  Data  from  Scenarios  2  and  4  were  selected 
because  they  both  used  sector  R62/63  and  had  a  different  number  of  aircraft.  Scenario  2,  was  run 
with  27  aircraft  and  two  controllers  and  Scenario  4  had  39  aircraft  and  three  controllers. 

An  examination  of  data  for  CVSM  with  27  aircraft  and  CVSM  with  39  aircraft  showed  that  the 
increase  in  traffic  did  not  result  in  an  increase  in  workload.  However,  RVSM  with  27  aircraft 
compared  to  RVSM  with  39  aircraft  showed  that  workload  and  operational  errors/deviations 
increased  with  more  traffic.  Finally,  an  inspection  of  CVSM  with  27  aircraft  and  RVSM  with  39 
indicated  that  workload  and  operational  errors/deviations  increased  during  RVSM.  This  data 
indicates  that  using  these  sectors  for  RVSM  transitions  does  not  necessarily  allow  an  increased 
traffic  flow  while  maintaining  current  workload  levels. 

5.  Conclusion 


Trends  in  the  data  indicate  that  controlling  traffic  in  the  studied  sectors  would  be  easier  under  the 
current  separation  compared  to  using  them  as  RVSM  transition  sectors.  The  Area  4  controllers’ 
ability  does  not  appear  to  be  affected  by  RVSM  when  traffic  levels  are  lower.  With  higher  levels 
of  traffic,  however,  the  controllers  experienced  higher  workload  and  had  more  operational  errors 
during  RVSM  compared  to  CVSM.  Errors  specifically  related  to  RVSM,  such  as  not 
transitioning  by  the  boundary,  occur  at  higher  traffic  levels.  Regardless  of  the  traffic  level,  it 
seems  that  with  RVSM  the  Area  4  controllers  assigned  more  altitudes  than  with  CVSM.  The 
effect  of  RVSM  on  the  Area  2  controller  was  clearer.  Tests  were  only  done  at  a  lower  traffic 
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level  and  results  show  that  Area  2  controllers  had  greater  difficulty  controlling  traffic  during 
RVSM  than  CVSM. 

For  Area  4,  contingency  operations  affected  important  controller  functions  such  as  the  ability  to 
maintain  separation  and  usually  increased  workload.  This  resulted  in  an  increased  number  of 
operational  errors.  Guidelines  to  handle  potential  complications  such  as  radar  outages  and  bad 
weather  need  to  be  developed  before  RVSM  can  be  safely  implemented. 

Overall,  the  introduction  of  RVSM  in  sectors  Rl,  R60,  R63,  R62/63,  R60/62/63  did  not  appear  to 
impact  the  controllers’  safety  priorities  or  controlling  strategies.  They  still  felt  that  maintaining 
separation  was  their  primary  safety  concern.  Transition  was  discussed  during  RVSM  runs  in 
addition  to  their  current  concerns.  Some  participants  indicated  that  they  made  minimal  changes 
to  their  controlling  strategy  to  accommodate  RVSM  transitions.  These  included  giving  more 
attention  to  altitude  assignments  and  becoming  more  aware  of  boundaries  and  non-radar  traffic. 

Participants  pointed  out  that  non-RVSM  participating  aircraft  might  create  a  problem  for 
controllers  because  they  will  have  to  enforce  the  2000-ft  separation  minima,  particularly  at 
crossing  fixes.  Non-participating  aircraft  may  be  penalized  severely  and  forced  to  use  an  altitude 
that  is  not  assigned  for  RVSM. 

Some  participants  felt  that  a  few  changes  needed  to  be  made  for  RVSM  implementation;  require 
all  aircraft  operating  in  this  space  to  be  RVSM  equipped,  increase  the  transition  airspace,  and 
have  access  to  more  reliable  radar.  However,  a  majority  of  participants  indicated  that  experience 
would  make  them  more  comfortable  with  RVSM  and  that  no  changes  would  be  necessary. 
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Acronyms 


7110.65 

A 

ARTCC 

ATC 

ATS 

D 

CRD 

CVSM 

DART 

DYSM 

EWG 

FAA 

FL 

FPL 

FT 

lATA 

ICAO 

MASPS 

MNPS 

NAT 

NATSPG 

NSC 

PVD 

R 

RGCSP 

RVSM 

SAR 

SIM 

TLS 

T/0 

WATRS 

VSM 

ZMA 

ZNY 


FAA  Air  Traffic  Controllers  Handbook 

Assistant  Controller 

Air  Route  Traffic  Control  Center 

Air  Traffic  Control 

Air  Traffic  Service 

Data  Controller 

Computer  Readout  Display 

Conventional  Vertical  Separation  Minima 

Data  Analysis  and  Reduction  Tool 

Dynamic  Simulation 

Experimentation  Working  Group 

Federal  Aviation  Administration 

Flight  level 

Full  Performance  Level 
Feet,  foot 

International  Air  Transport  Association 
International  Civil  Aviation  Organization 
Minimum  Aircraft  System  Performance  Specification 
Minimum  Navigation  Performance  Specification 
North  Atlantic 

North  Atlantic  System  Planning  Group 
National  Simulation  Capability 
Plan  View  Display 
Radar 

Review  of  the  General  Concept  of  Separation  Panel 
Reduced  Vertical  Separation  Minima 
System  Analysis  and  Recording 
Simulated 

Target  Level  of  Safety 

Technical  Observer 

Western  Atlantic  Track  Route  System 

Vertical  Separation  Minimum 

Miami  ARTCC 

New  York  ARTCC 
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Appendix  A 
Individual  Sector  Maps 


Figure  Al.  Individual  sector  map  for  R1 


Figure  A2.  Individual  sector  map  for  R60. 


A-2 


Gii4 


Figure  A3.  Individual  sector  map  for  R62. 


Figure  A4.  Individual  sector  map  for  R63. 


APPENDIX  B 


Background  Information  Form  and  Post-Run  Questionnaire 


BACKGROUND  INFORMATION  FORM 
INSTRUCTION  SHEET: 

This  form  is  to  be  completed  by  all  RVSM  Phase  IE  participants.  The  form  requests  general 
background  information  and  judgments  regarding  oceanic  control  practices. 

Your  name  will  not  be  listed  or  appear  in  any  reports  in  order  to  insure  your  anonymity  and  to 
encourage  unbiased  reporting.  Findings  will  be  reported  generically,  ex.  Controller  A,  B,  C,  etc. 

When  making  your  ratings,  please  consider  all  levels  of  the  scale.  You  are  encouraged  to  mark 
down  any  additional  comments  you  feel  are  important. 
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BACKGROUND  INFORMATION  FORM 


Controller  □  Observer  □  ID#: _ 

Date: _  Age: _ 

1.  How  long  have  you  actively  controlled  traffic  for  the  FAA? 

Years: _  Months: _ 

2.  How  many  of  the  past  12  months  have  you  actively  controlled  traffic  in  this  facility? 

Months: _ 

3.  Are  you  an  FPL? 

Yes  □  No  □ 

3A.  If  yes,  how  long  and  for  what  area? 

Years: _  Months: _  Area: _ 

3B.  If  no,  which  sectors  are  you  checked  out  on? 

1. _  2. _  3. _  4. _  5 _ 6._ 

4.  Name  the  last  3  facilities  (FAA)  you  have  worked  at  starting  with  your  current  assignment? 

1. _  2. _  3. _ 

5.  Have  you  had  experience  training  controllers  for  the  FAA? 

YcsD  Nod 

6.  Based  on  your  experience  with  high  traffic/workload  in  the  domestic  oceanic  or  WATRS  area: 

6A.  What  are  some  of  the  things  that  can  occur  that  could  cause  you  to  have  signi  ficant 
difficulties  in  maintaining  a  safe  and  expeditious  traffic  flow? 


6B,  Which  of  these  events  tend  to  occur  most  frequently? 
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BACKGROUND  INFORMATION  FORM  (cont.) 


7.  Please  rate  the  following  for  their  contribution  to  high  workload  levels  in  your  current  ATC  environment.  Circle 
your  response  using  the  scale  shown  below. 

y  -  Verx  Low  Conttibution"-  ^ '  Rkely  afifeoG  your  abiiftylo  complete  control &k$ ^ ^  Y;  *  Y 
Vei^  High  Contribution-  Frequendy  affects  jour  abiUtyjocomplejeoMto^^  ^  rti<£ 


A.  Random  Routing 
1 

Very  Low 
Contribution 


Very  High 
Contribution 


B.  Mix  of  Aircraft  Performance/Characteristics 
1  2  3 

Very  Low 
Contribution 


Very  High 
Contribution 


C.  Coordination  with  Foreign  Facilities 
1  2  3 

Very  Low 
Contribution 


Very  High 
Contribution 


D.  Coordination  with  Fellow  Controllers 


Very  Low 
Contribution 


Very  High 
Contribution 


E.  Flight  Strip  Printer  Speed 
1  2 
Very  Low 
Contribution 


Very  High 
Contribution 


F.  ARINC  Communications 
1  2 
Very  Low 
Contribution 


Very  High 
Contribution 


G.  Phone  System 
1 

Very  Low 
Contribution 


Very  High 
Contribution 


H.  Other.  Please  list  and  explain  below. 
Explain: 


CONTROLLER  POST-RUN  FORM 
INSTRUCTION  SHEET; 


This  form  is  to  be  completed  by  all  Air  Traffic  Controllers  participating  in  RVSM  Phase  HI.  The 
form  requests  information  regarding  your  overall  experiences  and  judgments  about  the  run  that 
you  just  completed. 

Your  name  will  not  be  listed  or  appear  in  any  reports  in  order  to  insure  your  anonymity  and  to 
encourage  unbiased  reporting.  Findings  will  be  reported  generically,  ex.  Controller  A,  B,  C,  etc. 

When  making  your  ratings,  please  consider  all  levels  of  the  scale.  You  are  encouraged  to  mark 
down  any  additional  comments  you  feel  are  important. 
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CONTROLLER  POST-RUN  FORM 


Date: _ 

Scenario: . 


ID#: _  Sector: 

Control  Position:  R  □  DO  AO 


1.  What  was  your  overall  worUoad  Jevd^urii^  this  pjob^^  shown  below. 

1  2  3  4  5  6  7  8 

Very  Low  Very  High 


2.  Rate  your  performance  for  the  following  factors  according  to  the  prescribed  scales: 

A.  Coordination 

Provided  complete  and  correct  information  for  clearances  and  estmates  m  a  timely  manner, 

1  2  3  4  5  6  7 

Very  Low 
Notes: 


Very  High 


B.  Maintenance  of  Situational  Awareness 

Addressed  all  aspects  ofjdrspace  and  aircraft 

r  . . 2^'"' . 

Poor 

Awareness 

Notes: 


notinffiKain  .^X’l ^ 

:aaa«aain^*aw^eness^.  ..ij'i  -iV 


Complete 

Awareness 


C.  Communication 

U^d  ATP  7110.65  specifications. 

^i^Ftat^iy'p]liowed^TP:7ili)*6^pecific^a^^''^^^^^ 


I  r  Always  fbito^ved  ATP  7 1  lOM  specificat|^:g 
12  3  4 

Rarely 
Notes: 


8 

Always 


D.  Priority  of  Actions 

Issued  control  instructions  in  prioritized,  structured  manner  as  specified  in  71 10.65. 


'}S^  V  YiSiUti  .  «<,  ■  x^v!*  Y. 


Very  Low 
Notes: 


Very  High 


CONTROLLER  POST-RUN  FORM  (cont.) 


E.  Flight  Strip  Management 

Marked  strips  accurately  while  performing  other  tasks;  kept  strips  current. 


;  ;:l£v ;’:, 

1  2  3 

Very  Low 
Notes: 


Very  High 


F.  Maintenance  of  Separation 

Adhered  to  separation  standards  as  specified  in  71 10.65. 

Gii)®' hot  maintain .separatioa';  •"  ■  .  ,.  ..,,  ,„.; . •  ,  ;,y,>  j;,.p.:'; 

Jisii^^aintainedseparatioii' .  1  "'•'''S;::  \- . 

1  2  3  4  5  6  7  8 


Very  Low 
Notes: 


Very  High 


G.  Computer  Entry 

Accurately  input  control  commands  into  key  panel. 
I  -  Had  great  diffictdty  inputting  commands 
|8  -  Easily  and  accufately  input  cormnands 

'  '  1  '2  3  '4' 

Many 

Errors 

Notes: 


No  Errors 


3.  Please  rate  the  following  simulation  characteristics  for  realism  compared  to  Miami’s  current  operations?  Please 
explain. 


A.  Traffic  Flow  (traffic  patterns,  traffic  volume,  etc.) 

1  2  3  4 

Very 

Unrealistic 

Explain: 


8 

Very 

Realistic 


CONTROLLER  POST-RUN  FORM  (cont.) 


B.  Physical  Environment  (PVDs,  lighting,  etc.) 

1  2  3  4  5  6 

Very 

Unrealistic 

Explain: 


7  8 

Very 
Realistic 


C.  Taskload  (#  communications,  #  requests,  etc.) 

1  2  3  4  5  6 

Very 

Unrealistic 

Explain: 


Very 

Realistic 


4.  What  was  your  primary  safety  concern  during  the  last  simulation  run? 


5.  Did  you  change  your  usual  control  strategies  in  any  way  while  working  with  RVSM  traffic? 
Yes  □  If  yes,  please  explain.  No  □  Not  an  RVSM  run  □ 

EXPLAIN: 


6.  Based  upon  your  experience  with  RVSM  in  the  last  run,  would  any  procedures  or  equipment  have  to  be  changed 
and/or  implemented  in  order  for  you  to  feel  comfortable  about  controlling  RVSM  traffic? 

Yes  □  If  yes,  please  explain.  No  □  Not  an  RVSM  run  □ 

EXPLAIN: 
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OBSERVER  FORM  (R  &  D) 
INSTRUCTION  SHEET: 


This  form  is  to  be  completed  by  all  Air  Traffic  Controllers  participating  in  RVSM  Phase  HI.  The 
form  requests  information  regarding  your  overall  experiences  and  judgments  about  the  run  that 
you  just  completed. 

Your  name  will  not  be  listed  or  appear  in  any  reports  in  order  to  insure  your  anonymity  and  to 
encourage  unbiased  reporting.  Findings  will  be  reported  generically,  ex.  Observer  A,  B,  C,  etc. 

When  making  your  ratings,  please  consider  all  levels  of  the  scale.  You  are  encouraged  to  mark 
down  any  additional  comments  you  feel  are  important. 
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OBSERVER  FORM  (R  &  D) 


Date: _ 

Scenario: . 


ID#: _ 

Start  Time: 


Controller  ID#:  R. 


Sector: . 


When  prompted,  please  rate  controllers  on  your  perception  of  their  workload.  In  addition,  note  the  time  you  were 
prompted. 


Interval  1 

R 

D 

Interval  2 

R 

D 

Interval  3 

R 

D 

Interval  4 

R 

D 

Notes: 


Time: 

2 

2 

Time: 

2 

2 

Time: 

2 

2 

Time: 

2 

2 


4 

4 

4 

4 

4 

4 

4 

4 


5 

5 

5 

5 

5 

5 

5 

5 


6 

6 

6 

6 

6 

6 

6 

6 


7 

7 

7 

7 

7 

7 

7 

7 


8 

8 

8 

8 

8 

8 

8 

8 


Time: _ 

Explain: 


Time: _ 

Explain: 


Time: _ 

Explain: 


Time: _ 

Explain: 
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OBSERVER  POST-RUN  FORM  (R  &  D  cont.) 


1.  What  did  you  perceive  to  be  the  controller’s  overall  workload  level  during  this  problem? 


t  all'it^ks  accwately^  • 

•  '$:■  ilimplete -jaU tasks'^" 

R  ~  1  2  3  4 

D  1  2  3  4 

Very  Low 


r”  *  n-  ' 
- '  'V.i, 


5  6  7  8  N\A 

5  6  7  8  N\A 

Very  High 


2.  Rate  the  controller’s  performance  during  this  problem  on  the  following  aspects: 


A.  Coordination 

Provided  complete  and  correct  information  for  clearances  and  estimates  in  a  timely  manner. 

;8-:'Al,%ayfg^yecorrecijDformaiion;j;:; 

R  1234567  SN’ 

D  1234567  SN’ 

Very  Low  Very  High 

Notes: 


Very  High 


B.  Maintenance  of  Situational  Awareness 
Addressed  all  aspects  of  airspace  and  aircraft. 
|]|fjEduid:no,ttomntainavvareness:^  ,v 
feyBasily  maintained  aw^eness  ; 

R  12  3  4 

D  12  3  4 

Poor 

Awareness 

Notes: 


Complete 

Awareness 


C.  Communication 

Used  ATP  71 10.65  specifications. 
^^^ar(il|'foljo^^  A’rP^iiO.65  speciffcations 
!8  -  ^#&ys  foliowMATP  7  ji  10.65  specifications 

R  . .  1  2  3  4 

D  12  3  4 

Rarely 

Notes: 


8 

Always 


.  rnuniy  oi 

sued  control  instructions  in  prioritized,  structured  manner  as  specified  in  7110.65. 

I^Kd^fiol^ollow  specified  pridrity^  - 

3*AIwayM6tlbwedsr)ecifiedDri6rity-'''  “'C-  K" 


D.  Priority  of  Actions 
Issued  control  instructions  in  prioritized, 
p^^d^n^foUow  sp^ified  priority 
fi  priprity;:  j,, 

R  1  2  3 

D  1  2  3 

Very  Low 

Notes: 


NVt 

N\A 


Very  High 


OBSERVER  POST-RUN  FORM  (R  &  D  cont.) 


E.  Flight  Strip  Management 

Marked  strips  accurately  while  performing  other  tasks;  kept  strips  current. 

R  ™  '^“'j '  2"  3  4  5  g  2  8 

D  1234567  8  N\A 

Very  Low  Very  High 

Notes: 


F.  Maintenance  of  Separation 

Adhered  to  sepyation  standards  as  specified  in  71 10.65 
®|^^^j||iteptain.si^ar^on  ' " ,  ”••; 


2  3  4  5  6  7  8  NU 


Very  Low 


Notes: 


8  N\A 

Very  High 


G.  Computer  Entry 

^^curately  jn()ut  control  commands  into  key  panel. 

■^BH":^eatdfflcuT^inpu&gcb  5  ^ 

iMM^.,ae0IS4elxlpiRUtgo|nm^  ^ 

R  12  3  4 

D  12  3  4 

Many  Errors 

Notes: 


No  Errors 


3.  Please  rate  the  following  simulation  characteristics  for  realism  compared  to  Miami’s  current  operations?  Please 
explain. 


A.  Traffic  Flow  (traffic  patterns,  traffic  volume,  etc.) 

12  3  4 

Very 

Unrealistic 

Explain: 


B.  Physical  Environment  (PVDs,  lighting,  etc.) 

12  3  4 

Very 

Unrealistic 

Explain: 


8 

Very 

Realistic 


8 

Very 

Realistic 


OBSERVER  POST-RUN  FORM  (R  &  D  cont.) 


C.  Taskload  (#  communications,  #  requests,  etc.) 

12  3  4 

Very 

Unreal!  sti 

5 

6 

7 

8 

Very 

Realistic 

c 

Explain: 

4.  What  was  your  primary  safety  concern  during  the  last  simulation  run? 


5.  Did  the  controller  change  her/his  control  strategies  used  during  CVSM  in  order  to  work  with  RVSM  traffic? 
Yes  n  If  yes,  explain  what  she/he  did.  No  □  Not  an  RVSM  run  □ 

Explain  “R”:  _ 


Explain  “D”: 


6.  Based  upon  your  experience  with  RVSM  in  the  last  run,  would  any  procedures  or  equipment  have  to  be  changed 
and/or  implemented  in  order  for  you  to  feel  comfortable  about  controlling  RVSM  traffic? 

Yes  □  If  yes,  please  explain.  No  □  Not  an  RVSM  run  □ 

Explain: _ 
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SUPPORT  TEAM  FORM 
INSTRUCTION  SHEET: 


1.  This  form  is  to  be  used  by  NSC  support  personnel. 

2.  Prompt  controllers  every  15  minutes  to  give  their  workload  level.  In  addition,  note  the  time 
the  controller  was  asked  to  respond. 

3.  Note  the  number  of  communications  by  each  controller  with  a  hash  mark. 
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SUPPORT  TEAM  FORM 


Date: _ _  Controller  ID#:  R _  D _  A _ 

Scenario: _  Sector: _ 

Controller  Self  Reported  Workload  Ratings. 

Interval  1  Time: _ 

R123456  7  8 

D123456  7  8 

A123456  7  8 


Interval  2  Time: _ 

R123456  7  8 

D123456  7  8 

A123456  7  8 

COMMUNICATIONS 


Interval  3 

R 

Time: 

1  2 

3 

4 

5 

6 

7 

8 

D 

1  2 

3 

4 

5 

6 

7 

8 

A 

1  2 

3 

4 

5 

6 

7 

8 

Appendix  C 

Controller  and  Observer  Background  Summaries 


Table  Cl.  Controller  Background  Information  Summary 


33.1 

3.45 

6.62 

3.14 

4.28 

3.28 

Frequency  Mean  Std.  Dev 


38 


Participants 


Age  (Years) 


Years  Active  for  FA  A 


Years  as  FPL  Status 


Non-FPL  Status 


Number  of  Last  12  Months  Active 


Have  Training  Experience 


No  Training  Experience 


Work  in  Area  2 


Work  in  Area  4 


Worked  Facilities  Other  Than  ZMA 


Table  C2.  Observer  Background  Information  Summary 


11.96  0.36 


Participants 


Age  (Years) 


Years  Active  for  FA  A 


Years  as  FPL  Status 


Non-FPL  Status 


Number  of  Last  12  Months  Active 


Have  Training  Experience 


No  Training  Experience 


Work  in  Area  2 


Work  in  Area  4 


Worked  Facilities  Other  Than  ZMA 


33.3 

3.8 

8.55 

3.8 

6.07 

3.01 

Table  C3.  Controller  and  Observer  Ratings  for  Effect  of  ATC  Factors  on  Workload  in  Current 

ATC  Environment 


ATC  Factor  ^ 

Area  2 

Area  4 

Mean 

Std.  Dev. 

Mean 

Std.  Dev. 

Random  Routing 

2.87 

2.29 

2.85 

1.68 

Mix  of  Aircraft  Performance/Characteristics 

4.40 

2.03 

4.32 

1.79 

Coordination  with  Foreign  Facilities 

2.07 

1.58 

5.90 

1.50 

Coordination  with  Fellow  Controllers 

3.27 

1.62 

2.83 

1.41 

Flight  Strip  Printer  Speed 

2.13 

1.64 

3.02 

1.84 

ARINC  Communications 

2.13 

0.99 

2.22 

1.21 

Phone  System 

3.20 

1.52 

4.44 

2.32 

^  Ratings  made  on  a  scale  from  1  to  8 
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Appendix  D 

Statistical  Values  of  Measures 


SCENARIO  1 
Controller  Post-Run  Data 


C^AWARE 


R.AWARE 


E.AWARE 


C.COMPUT 


R_COMPUT 


E.COMPUT 


C_COMTOT 


R^COMTOT 


E_COMTOT 


C^COORD 


R.COORD 


E_COORD 


C_PRIOR 


R_PRIOR 


E_PRIOR 


C_SEPAR 


R_SEPAR 


C.STRIPS 


R.STRIPS 


E.STRIPS 


C_WORK 


E_WORK 


C.AWARE 


R.AWARE 


E„AWARE 


C^COMM 


R^COMM 


E^COMM 


C_.COMPUT 


R  COMPUT 


Position 

R 

D 

Mean 

Std.  Dev. 

Mean 

Std.  Dev. 

7.60 

0.55 

7.40 

0.89 

7,00 

1.00 

7.00 

1.00 

4.33 

1.53 

5.00 

0.00 

7.60 

0.55 

7.80 

0.45 

7.40 

0.55 

7.80 

0.45 

6.00 

1.00 

6.33 

0.58 

6.80 

1.10 

7.60 

0.55 

7.00 

1.00 

7.40 

0.55 

6.00 

1.00 

7.00 

0.00 

111.80 

23.79 

75.20 

13.81 

111.60 


E_COMPUT 


C_COMTOT 


R_COMTOT 


E_COMTOT 


C_COORDS 


R_COORDS 


E.COORDS 


C_PRIOR 


R_PRIOR 

E_PRIOR 


C_SEPAR 


E_SEPAR 


C_STRIPS 


E_STRIPS 


C_WORK 


R_WORK 
E  WORK 


R_COMPUT 
E  COMPUT 


E_COMTOT 


C.COORD 

R_COORD 


E_COORD 


C.PRIOR 


R.PRIOR 

E.PRIOR 


C_SEPAR 


R  SEPAR 


7.00 


96.00 


97.80 


134.67 


7.60 


7.60 


4.67 


7.60 


7.40 

6.67 


8.00 


7.40 


4.00 


7.60 


5.40 


3.33 


1.20 


2.00 

~5.00 


Group  Total 

C_AWARE 

7.60 

R_AWARE 

7.30 

E„AWARE 

4.33 

C.COMM 

7.60 

R_COMM 

7.40 

E_COMM 

5.67 

C_COMPUT 

7.30 

136.67 


7.70 

7.40 


5.17 


7.70 


7.00 


6.17 


7.70 


7.20 


1.00 

7.33 

14.27 

31.20 

19.28 

36.20 

16.35 

93.67 

0.55 

6.80 

0.55 

7.40 

3.51 

0.55 

7.80 

0.55 

7.80 

1.15 

6.33 

0.00 

8.00 

0.89 

8.00 

2.00 

6.00 

0.55 

7.80 

2.61 

7.60 

2.31 

6.33 

0.45 

1.40 

1.22 

2.20 

1.73 

5.67 

1.10 


0.58 


0.67 


0.82 


0.63 


0.42 


0.48 


0.75 


0.48 


0.48 


0.41 


26.58 

25.11 


32.84 


1.89 


0.84 


1.03 


0.70 


0.48 


1.03 


0.32 


0.32 


7.17 


63.6 


67.00 


101.50 


7.20 


6,50 


8.00 


7.7 


5.00 


7.70 


6.50 


4.83 


1.30 


2.10 


5.33 


7.65 


7.30 


4.67 


7.70 


7.55 


6.25 


7.50 


7.45 


6.83 


78.55 

79.55 


115.17 


7.35 


7.40 


5.75 


7.65 


7.37 


6.25 


7.80 


7.55 


0.75 


36.62 


36.79 


37.49 


1.87 


0.71 


2.59 


0.48 


0.52 


1.22 


0.00 


0.67 


1.79 


0.48 


2.12 


2.40 


0.48 


1.10 


1.21 


0.59 


0.80 


1.07 


0.47 


0.51 


1.14 


0.76 


0.69 


0.83 


34.73 


34.44 


33.41 


1.39 


0.75 


1.91 


0.59 


0.76 


0.97 


0.52 


0.76 
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4.67 
7^40 

R_STRIPS  Jm 

E_STRIPS  150 

CJWORK  ^ 

R_WORK  ^ 

E_WORK  133 


E.SEPAR 

CISTRIPS 


Mean 

SECTOR  R60  C.AWARE  7.60 

R_AWARE  7.20 

E.AWARE  7.00 

7.80 
7^0~ 
1~,50~ 
7,40 
7.20 
7.00 

C_COMTOT  114720 

R_COMTOT  117.80 

E_COMTOT  136.00 

C.COORD  7.60 

R_COORD  7.60 

E_COORD  7.50 

C_PRIOR  7^0 


C_COMM 

rIcomm 

eIcomm 

C.COMPUT 

R_COMPUT 

eIcomput 


R_PRIOR  7.60 

E.PRIOR  ^ 

C_SEPAR  ^ 

R_SEPAR  ^ 

E_SEPAR  ^ 


C_WORK  2.80 


R.WORK  2.60 


0.63 

5.33 

1.56 

0.52 

7.50 

0.61 

1.93 

7.40 

0.52 

6.60 

1.60 

1.97 

1.03 

4.92 

2.11 

1.25 

1.05 

2.00 

1.12 

1.81 

3.00 

1.56 

1.65 

1.37 

5.50 

1.38 

5.42 

1.31 

SCENARIO  2 
3ller  Post-Run  Data 


Position 

Group  Test 

R 

D 

Mean 

Std.  Dev. 

Std.  Dev. 

Mean 

Std.  Dev. 

0.55 

7.60 

0.55 

7.60 

0.52 

1.10 

0.55 

7.30 

0.82 

0.00 

0.71 

6.75 

0.50 

0.45 

0.55 

imm 

0.48 

0.89 

7.80 

0.45 

0.67 

0.71 

7.50 

0.55 

7.60 

1.10 

7.60 

0.55 

7.40 

0.84 

0.00 

7.50 

0.71 

7.25 

0.50 

13.63 

58.20 

32.95 

86.20 

37.90 

43.40 

21.41 

80.60 

42.97 

78.00 

52.33 

107.00 

48.55 

0.45 

7.70 

0.48 

0.55 

7.40 

0.55 

7.50 

0.53 

0.71 

SB 

7.25 

0.50 

0.45 

7.80 

0.42 

0.89 

7,60 

0.55 

7.60 

0.71 

7.00 

0.00 

6.75 

0.00 

8.00 

0.00 

8.00 

0.00 

0.00 

8.00 

0.00 

8.00 

0.00 

0.71 

6.50 

0.71 

6.50 

0.58 

1.22 

0.89 

7.20 

1.03 

2.07 

6.60 

1.58 

0.00 

7.00 

0.00 

6.50 

0.58 

1.48 

1.80 

1.79 

1.64 

1.14 

2.00 

1.22 

1.16 
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E.WORK 


R62/63  C_AWARE 


R.AWARE 


E_AWARE 


C  COMM 


R_COMM 

E_COMM 

C_COMPUT 

7.60 

R_COMPUT 

7.60 

E.COMPUT 

7.00 

C_COMTOT 

106.40 

Group  Total 


R.COMTOT 


E_COMTOT 


C_COORD 


R_COORD 


E_COORD 


C_PRIOR 

R_PRIOR 


E_PRIOR 

C_SEPAR 


R.SEPAR 

E_SEPAR 


C_STRIPS 

R_STRIPS 


E_STRIPS 

C_WORK 


R_WORK 


E_WORK 

C_AWARE 


R_AWARE 

E„AWARE 


C.COMM 

R_COMM 


E_COMM 

C_COMPUT 


R_COMPUT 


E_COMPUT 


C.COMTOT 


R  COMTOT 


109.80 


142.50 


7.60 


7.60 


6.00 


7.60 

7.40 


7.00 

7.80 


7.60 
"  ^50 


6.40 

7.00 


4.50 

"3^0" 


3.60 


5.00 

7.50 


7.10 

615 


7.80 

7.70 


7.25 

7.50 


7.40 


7.00 


110.30 


113.80 


0.71 

6.75 

0.96 

0.55 

7.50 

0.53 

0.89 

7.50 

0.71 

0.71 

7.25 

0.96 

0.45 

7.80 

0.42 

0.45 

7.70 

0.48 

0.71 

7.00 

0.82 

E.COMTOT 

139.25 

C_COORD 

7.60 

R_COORD 

7.60 

E_COORD 

6.75 

C.PRIOR 

7.70 

R_PRIOR 

7.50 

E_PRIOR 

6.75 

C.SEPAR 

7.90 

R^SEPAR 

7.80 

E_SEPAR 

6.50 

C_STRIPS 

6.70 

R_STRIPS 

6.90 

E__STRIPS 

5.25 

C_WORK 

3.00 

R_WORK 

3.10 

E  WORK 


C.AWARE 

7.40 

R_AWARE 

7.40 

C_COMM 

R.COMM 

7.20 

R.COMPUT 

7.20 

C_COMTOT 

104.40 

R_COMTOT 

97.20 

C.COORD 

7.80 

R_COORD 

7.40 

C_PRIOR 

7.60 

R_PRIOR 

7.60 

C_SEPAR 

8.00 

R.SEPAR 

7.60 

C_STRIPS 

7.20 

R_STRIPS 

7.20 

C^WORK 

2.40 

SCENARIO  3 
Controller  Post-Run  Data 

Position 


Std.  Dev. 


0.89 


0.55 


0.00 


0.00 
IJO  ' 


Group  Total 


Mean 


Std.  Dev. 


Std.  Dev, 


Group  Total 


R_WORK 

2.40 

C_AWARE 

6.20 

R.AWARE 

5.00 

C_COMM 

7.20 

R_COMM 

6.20 

C.COMPUT 

5.80 

R.COMPUT 

5.60 

C.COMTOT 

133.40 

R.COMTOT 

C.COORD 

R_COORD 

4.80 

C_PRIOR 

6.40 

R_PRIOR 

6.20 

C_SEPAR 

7.00 

R_SEPAR 

5.00 

C.STRIPS 

4.60 

R_STRIPS 

4.20 

C_WORK 

2.80 

R_WORK 

4.00 

C^AWARE 

6.80 

R_AWARE 

C„COMM 

7.60 

_ 

R_COMM 

7.10 

C_COMPUT 

6.50 

R_COMPUT 

6.40 

C.COMTOT 

118.90 

R_COMTOT 

119.80 

C_COORD 

7.10 

R_COORD 

6.10 

C.PRIOR 

7.00 

R_PRIOR 

6.90 

C.SEPAR 

7.50 

R_SEPAR 

6.30 

C_STRIPS 

5.90 

R.STRIPS 

5.70 

C_WORK 

2.60 

R_WORK 

3.20 

1.95 

6.30 

0.55 

6.60 

1.00 

7.60 

1.41 

6.00 

0.89 

6.80 

2.17 

6.80 

0.84 

2.80 

6.70 

0.67 

6.60 

0.70 
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SCENARIO  4 


Controller  Post-Run  Data 


Position 

Group  Total 

R 

D 

A 

Mean 

Std.  Dev. 

Mean 

Std.  Dev. 

Mean 

Std.  Dev. 

Mean 

Std.  Dev. 

C_AWARE 

4.80 

2.17 

6.60 

1.67 

6.60 

2.07 

6.00 

2.04 

R.AWARE 

4.60 

1.95 

5.80 

1.10 

5.40 

1.82 

5.27 

1.62 

E_AWARE 

3.50 

1.00 

5.50 

2.38 

5.25 

2.22 

4.75 

2.01 

C_COMM 

6.00 

2.24 

7.80 

0.45 

7.25 

0.96 

7.00 

1.57 

R.COMM 

7.00 

0.71 

7.80 

0.45 

7.60 

0.55 

7.47 

0.64 

E_COMM 

5.00 

1.15 

6.25 

1.71 

7.25 

0.96 

6.17 

1.53 

C_COMPUT 

6.60 

0.55 

7.40 

0.89 

7.67 

0.58 

7.15 

0.80 

R.COMPUT 

6.20 

0.84 

7.20 

0.84 

7.50 

0.71 

6.83 

0.94 

E_COMPUT 

4.25 

1.26 

7.25 

0.96 

7.00 

1.41 

6.17 

1.80 

C_COMTOT 

166.60 

31.16 

133.40 

45.47 

17.80 

11.12 

105.93 

72.54 

R_COMTOT 

171.40 

50.26 

140.60 

22.68 

29.80 

10.71 

113.93 

69.73 

E_COMTOT 

209.25 

48.26 

101.75 

30.97 

78.75 

29.75 

129.92 

68.32 

C.COORD 

5.80 

2.17 

7.80 

0.45 

7.75 

0.50 

7.07 

1.59 

R_COORD 

6.40 

0.89 

6.80 

1.30 

6.40 

0.55 

6.53 

0.92 

E_COORD 

4.25 

2.22 

6.75 

1.89 

6.50 

1.29 

5.83 

2.04 

C_PRIOR 

6.80 

0.84 

7.60 

0.55 

7.33 

1.15 

7.23 

0.83 

R.PRIOR 

6.20 

1.48 

7.40 

0.55 

7.25 

0.96 

6.93 

1.14 

E_PRIOR 

4.75 

0.96 

7.25 

0.50 

6.25 

2.22 

6.08 

1.68 

C_SEPAR 

7.40 

0.89 

8.00 

0.00 

7.50 

0.71 

7.67 

0.65 

R.SEPAR 

7.00 

1.22 

7.20 

0.84 

7.50 

0.71 

7.17 

0.94 

E_SEPAR 

3.00 

1.15 

6.50 

1.73 

5.50 

3.54 

4.90 

2.38 

C_STRIPS 

5.20 

1.48 

6.80 

0.84 

7.75 

0.50 

6.50 

1.45 

R_STRIPS 

4.80 

2.49 

5.80 

1.64 

6.25 

2.22 

5.57 

2.06 

E_STRIPS 

2.00 

0.82 

4.75 

2.87 

5.00 

2.94 

3.92 

2.61 

C_WORK 

4.00 

2.35 

4.00 

2.45 

2.00 

1.73 

3.33 

2.26 

R_WORK 

5.40 

1.52 

4.20 

1.48 

4.60 

1.95 

4.73 

1.62 

E_WORK 

7.75 

0.50 

5.50 

3.00 

4.75 

2.06 

6.00 

2.34 

SCENARIO  1 
Controller  Interval  Data 


Position 

Group  Total 

R 

D 

Mean 

Std.  Dev. 

Mean 

Std.  Dev. 

Mean 

Std.  Dev. 

SECTOR 

R63 

C„WORK 

1.70 

0.98 

1.75 

1.07 

1.73 

1.01 

R_WORK 

2.15 

1.35 

1.90 

1.12 

2.03 

1.23 

E.WORK 

3.83 

2.33 

3.75 

1.82 

3.79 

2.04 

R60 

C_WORK 

1.35 

0.59 

1.20 

0.41 

1.27 

0.51 

R_WORK 

1.65 

1.18 

1.40 

0.82 

1.53 

1.01 

E_WORK 

3,67 

2.10 

3.50 

2.07 

3.58 

2.04 

Group  Total 

C_WORK 

1.53 

0.82 

1.48 

0.85 

1.50 

0.83 

REWORK 

1.90 

1.28 

1.65 

1.00 

1.78 

1.15 

E^WORK 

3.75 

2.17 

3.63 

1.91 

3.69 

2.02 
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SCENARIO  2 
Controller  Interval  Data 


Position 

R 

Mean 

Std.  Dev. 

Mean 

Sector 

R60 

C_WORK 

2.40 

1.82 

2.30 

R_WORK 

2.45 

1.67 

2.15 

E^WORK 

2.50 

1.51 

2.88 

R62/63 

C_WORK 

2.15 

1.90 

2.25 

R_WORK 

3.10 

2.00 

3.15 

E_WORK 

3.63 

1.85 

2.75 

Group  Total 

C_WORK 

2.28 

1.84 

2.28 

R_WORK 

2.78 

1.85 

2.65 

E_WORK 

3.06 

1.73 

2.81 

SCENARIO  3 
Controller  Interval  Data 


Position 

R 

Mean 

Std.  Dev. 

Mean 

Sector 

R60/62/63 

C_WORK 

1.85 

1.04 

2.55 

R_WORK 

2.25 

1.37 

2.10 

R1 

C_WORK 

2.10  1 

1.21 

R_WORK 

3.05 

1.82 

Group  Total 

C_WORK 

1.98 

1.12 

2.30 

R_WORK 

2.65 

1.64 

2.38 

SCENARIO  4 
Controller  Interval  Data 


Position 

R 

D 

Mean 

Std.  Dev. 

Mean 

Std.  Dev. 

Mean 

C_WORK 

3.90 

2.13 

3.95 

2.31 

2.00 

E_AWARE 


C_COMM 


R_COMM 


E_COMM 


C_COMPUT 


R_COMPUT 


E_COMPUT 


R_COORD 

E„COORD 


C„STRIPS 


R_STRIPS 


E_STRIPS 


C_PRIOR 


R_PRIOR 


E_PRIOR 


C„SEPAR 


R.SEPAR 


E_SEPAR 


C_WORK 


R_WORK 


E_WORK 


C_AWARE 


R.AWARE 


E.AWARE 


C  COMM 


R.COMPUT 


E.COMPUT 


C^COORD 


R.COORD 


Mean 


SCENARIO  1 
T/0  Post-Run  Data 


Control  Position 

_ 

Std.  Dev.  Mean 


7.80 


7. 


m 


Group  Total 


Std.  Dev. 


7.50 

0.58 

6.60 

1.14 

6.67 

1.53 

7.40 

0.55 

7.20 

1.10 

7.00 

1.00 

7.60 

0.55 

7.40 

0.55 

7.67 

0.58 

7.80 

0.45 

7.60 

0.89 

7.33 

0.58 

7.20 

0.84 

Std.  Dev. 


0.45 

7.67 

0.50 

1.22 

6.80 

1.14 

1.53 

6.50 

1.38 

0.89 

7.40 

0.70 

1.41 

7.10 

1.20 

1.00 

7.00 

0.89 

0.45 

7.70 

0.48 

0.55 

7.40 

0.52 

0.58 

7.67 

0.52 

0.00 

7.90 

0.32 

1.00 

7.30 

0.95 

0.58 

7.00 

0.63 

0.55 

7.30 

0.67 

1.79 

5.30 

2.45 

2.08 

5.50 

1.97 

0.89 

7.30 

1.06 

0.50 

1.32 

1.15 

6.67 

1.03 

0.45 

7.89 

0.33 

0.89 

7.40 

1.35 

0.00 

7.83 

0.41 

1.34 

2.40 

1.26 

1.22 

3.80 

1.03 

0.58 

6.50 

1.05 

0.55 

7.30 

0.67 

0.55 

7.40 

0.97 

2.31 

4.67 

1.75 

0.45 

7.70 

0.48 

2.24 

7.30 

1.57 

0.00 

6.83 

0.41 

1.10 

7.20 

0.92 

0.00 

8.00 

0.00 

0.58 

7.33 

0.52 

0.55 

7.40 

0.84 

1.41 

7.38 

1.06 

Group  Total 


E.COORD 

6.67 

CASTRIES 

7.60 

R_STRIPS 

7.60 

E.STRIPS 

4.33 

Qggggjlllll 

7.80 

7.80 

E.PRIOR 

7.00 

C_SEPAR 

8.00 

R_SEPAR 

8.00 

E_SEPAR 

4.67 

C^WORK 

2.00 

R_WORK 

2.20 

E_WORK 

6.00 

C„  AWARE 

7.33 

R_AWARE 

6.90 

E_AWARE 

5.67 

C_COMM 

7.50 

R_COMM 

7.40 

E_COMM 

6.83 

C_COMPUT 

7.40 

R_COMPUT 

7.70 

E_COMPUT 

7.50 

C.COORD 

7.50 

R_COORD 

7.67 

E_COORD 

7.00 

0.58 

6.33 

0.55 

7.60 

0.55 

7.20 

2.08 

4.67 

0.45 

7.60 

0.45 

7.80 

1.00 

7.00 

0.00 

8.00 

8.00 

■EIB 

5.33 

1.22 

2.20 

0.84 

2.40 

1.00 

5.33 

0.71 

7.60 

1.20 

7.30 

1.75 

5.50 

R_STRIPS 

6.20 

2.57 

E_.STRIPS 

4.83 

2.04 

C_PRIOR 

7.50 

0.97 

R_PRIOR 

7.40 

1.26 

E.PRIOR 

6.83 

0.98 

C_SEPAR 

8.00 

0.00 

R.SEPAR 

7.60 

1.26 

E.SEPAR 

6.17 

2.23 

C_WORK 

2.20 

1.23 

R_WORK 

2.90 

1.10 

E_WORK 

6.17 

1.17 

0.44 

7.58 

0.98 

6.83 

0.32 

7.95 

0.63 

7.70 

2.34 

6.42 

1.25 

2.25 

1.55 

3.05 

1.55 

6.08 
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SCENARIO  2 
T/0  Post-Run  Data 


Control  Position 

R 

D 

Mean  Std.  Dev. 

Mean  Std.  Dev. 

SECTOR 


C.AWARE 


R_AWARE 


E.AWARE 


C_COMM 


R_COMM 


E.COMM 


C_COMPUT 


R_COMPUT 


E_COMPUT 


C.COORD 


R_COORD 


E.COORD 


C  STRIPS 


R.STRIPS 

E.STRIPS 

C_PRIOR 

R_PRIOR 

R_WORK 

E.WORK 

R62/63 

C^AWARE 

R.AWARE 

E.AWARE 

R_COMPUT 


E_COMPUT 


C_COORD 


R  COORD 


7.40 

1.34 

6.80 

1.30 

4.00 

4.24 

7.60 

0.55 

0.55 

6.50 

2.12 

7.80 

0.45 

7.80 

0.45 

7.50 

0.71 

7.60 

0.89 

7.40 

0.89 

6.40 

0.89 

2.83 

7.80 

0,45 

8.00 

0,00 

5.50 

2.12 

8.00 

0.00 

2.50 

0.71 

3.80 

1.64 

3.60 

2.07 

6.50 

0.71 

7.40 

0.89 

7.80 

0.45 

6.50 

0.71 

7.40 

0.89 

7.20 

0.84 
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E_COORD 

7.50 

0.71 

7.50 

0.71 

0.58 

C_STRIPS 

6.60 

1.67 

6.00 

2.45 

6.30 

2.00 

R_STRIPS 

7.20 

0.84 

6.60 

1.52 

6.90 

E.STRIPS 

7.00 

1.41 

7.00 

1.41 

7.00 

C.PRIOR 

7.40 

0.89 

7.20 

0.84 

0.82 

R_PRIOR 

7.20 

0.84 

7.60 

0.55 

0.70 

E_PRIOR 

7.50 

0.71 

0.71 

0.58 

C_SEPAR 

7.60 

0.89 

7.80 

0.45 

0.67 

R.SEPAR 

7.60 

E.SEPAR 

7.50 

7.50 

0.58 

C_WORK 

4.20 

2.28 

5.00 

2.45 

4.60 

2.27 

R_WORK 

3.60 

1.95 

4.00 

2.12 

3.80 

1.93 

E_WORK 

3.00 

1.41 

0.71 

3.25 

0.96 

Group  Total 

C_AWARE 

7.30 

0.95 

mi^mi 

1.45 

7.10 

1.21 

R_AWARE 

7.20 

1.03 

1.43 

6.90 

1.25 

E_AWARE 

5.75 

1.89 

5.50 

3.00 

5.63 

2.33 

C_COMM 

7.50 

0.71 

7.50 

0.71 

7.50 

0.69 

R_COMM 

7.40 

0.70 

7.60 

0.52 

7.50 

0.61 

E_COMM 

7.00 

1.41 

7.25 

1.50 

7.13 

1.36 

C_COMPUT 

7.40 

0.84 

7.60 

0.52 

7.50 

0.69 

R_COMPUT 

7.70 

0.48 

7.80 

0.42 

7.75 

0.44 

E_COMPUT 

7.00 

0.82 

7.25 

0.50 

7.13 

0.64 

C_COORD 

7.40 

nmgiiiii 

1.63 

7.20 

1.28 

R.COORD 

7.30 

0.67 

1.29 

7.10 

1.02 

E.COORD 

6.25 

1.71 

7.50 

0.58 

6.88 

1.36 

C„STRIPS 

6.60 

1.26 

1.90 

6.55 

1.57 

R_STRIPS 

6.80 

0.92 

6.80 

1.55 

6.80 

1.24 

E_STRIPS 

6.50 

1.91 

7.00 

1.15 

6.75 

1.49 

C_PRIOR 

7.60 

0.70 

7.40 

0.84 

7.50 

0.76 

R_PRIOR 

7.60 

0.70 

7.70 

0.48 

7.65 

0.59 

E_PRIOR 

6.50 

1.73 

6.75 

1.26 

6.63 

1.41 

C.SEPAR 

7.70 

0.42 

7.75 

0.55 

R_SEPAR 

7.80 

0.32 

7.85 

0.37 

E_SEPAR 

5.00 

2.94 

6.00 

3.37 

5.50 

2.98 

C_WORK 

4.00 

1.89 

4.50 

2.12 

4.25 

1.97 

R_WORK 

3.60 

1.90 

3.90 

1.97 

3.75 

1.89 

E_WORK 

4.75 

2.22 

4.50 

1.29 

4.63 

1.69 
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SECTOR  R60/62/63  C_AWARE 


R_AWARE 


C_COMM 


R_COMM 


C_COMPUT 


R_COMPUT 


C„COORD 


R.COORD 


C__STRIPS 


R_STRIPS 

C_PRIOR 


R„PRIOR 


C_SEPAR 


R.SEPAR 


C_WORK 


R.WORK 

C.AWARE 


R_AWARE 


C_COMM 

R_COMM 


C_COMPUT 

R_COMPUT 


C.COORD 

R_COORD 


C_STRIPS 

R„STRIPS 

C_PRIOR 


R„PRIOR 

c_separ“ 


R_SEPAR 


C_WORK 


I  R_WORK 


SCENARIO  3 
T/0  Post-Run  Data _ 

Control  Position 


Std.  Dev. 


0.45 


0.55 


0.27 


0.27 


0.42 


0.84 


0.42 


0.35 


0.65 


Group  Total 


Mean 

Std.  Dev. 

7.00 

1.00 

7.10 

1.19 

7.20 

0.67 

7.22 

0.41 

7.50 

0.35 

7.20 

0.84 

7.20 

0.67 

7.30 

0.76 

6.90 

1.08 

6.80 

0.91 

7.50 

0.35 

7.40 

0.55 

7.50 

1.12 

7.70 

0.45 

4.30 

2.08 

3.80 

1.79 

7.40 

0.55 

5,80 

1.79 

7.80 

0.45 

5.40 

3.21 

Group  Total 

C_AWARE 

7.20 

0.70 

R_AWARE 

6.80 

1.32 

6.45 

1.59 

6.63 

1.43 

C_COMM 

7.55 

0.44 

7.50 

7.53 

0.53 

R_COMM 

6.95 

1.44 

6.31 

2.36 

6.63 

1.93 

C.COMPUT 

7.50 

7.65 

0.41 

7.58 

0.54 

R_COMPUT 

7.25 

0.70 

7.33 

0.73 

C.COORD 

7.60 

0.63 

R_COORD 

7.45 

7.35 

0.70 

C_STRIPS 

7.05 

7.25 

1.01 

7.15 

0.97 

R_STRIPS 

6.75 

1.06 

6.78 

0.92 

C_PRIOR 

7.30 

mom 

0.63 

7.33 

R_PRIOR 

6.85 

IBDii 

0.88 

6.98 

1.15 

C_SEPAR 

7.60 

7.45 

0.96 

7.53 

0.88 

R_SEPAR 

6.95 

2.03 

1.79 

C_WORK 

3.45 

1.81 

1.58 

R_WORK 

3.80 

1.40 

4.10 

1.52 

3.95 

1.43 

SCENARIO  4 
T/0  Post-Run  Data 


ControS  Position 

Group  Total 

R 

D 

A 

Mean 

Std.  Dev. 

Mean 

Std.  Dev. 

Mean 

Std.  Dev. 

Mean 

Std-  Dev. 

C_AWARE 

6.20 

0.97 

5.80 

1.92 

7.25 

0.96 

6.36 

1.42 

R.AWARE 

5.50 

1.41 

5.60 

1.67 

6.80 

1.10 

5.97 

1.45 

E_AWARE 

3.63 

1.11 

3.75 

2.22 

5.25 

2.63 

4.21 

2.04 

C_COMM 

7.20 

0.67 

7.40 

0.89 

7.75 

0.50 

7.43 

0.70 

R_COMM 

7.10 

0.65 

7.40 

0.89 

7.60 

0.55 

7.37 

0.69 

E_COMM 

6.13 

1.93 

7.50 

0.58 

7.75 

0.50 

7.13 

1.32 

C_COMPUT 

6.80 

1.04 

6.80 

1.64 

7.33 

1.15 

6.92 

1.24 

R_COMPUT 

7.10 

0.42 

7.40 

0.89 

7.50 

0.58 

7.32 

0.64 

E_COMPUT 

5.25 

1.26 

7.25 

0.96 

7.67 

0.58 

6.64 

1.43 

C_COORD 

6.90 

0.42 

7.20 

1.10 

8.00 

0.00 

7.32 

0.80 

R_COORD 

6.70 

1.04 

6.40 

1.14 

7.20 

0.84 

6.77 

1.00 

E_COORD 

5.50 

1.29 

5.75 

2.22 

7.25 

0.96 

6.17 

1.64 

C_STRIPS 

5.80 

0.97 

4.80 

2.77 

7.00 

1,41 

5.79 

1.99 

R_STRIPS 

5.10 

1.56 

4.40 

2.61 

7.00 

1.41 

5.39 

2.12 

E„STRIPS 

3.75 

1.66 

3.50 

3.00 

6.00 

1.73 

4.27 

2.32 

C_PRIOR 

7.00 

0.35 

7.00 

1.00 

7.75 

0.50 

7.21 

0.73 

R_PRIOR 

7.00 

0.61 

7.00 

0.71 

7.50 

0.58 

7.14 

0.63 

E_PRIOR 

5.00 

1.58 

5.25 

2.63 

6.33 

2.08 

5.45 

2.01 

D-14 


C.SEPAR 

7.00 

1.00 

6.00 

2.12 

7.33 

1.15 

6.69 

1.55 

R_SEPAR 

6.50 

1.46 

6.75 

0.96 

8.00 

0.00 

6.96 

1.20 

E.SEPAR 

3.25 

1.19 

5.00 

2.58 

6.67 

2.31 

4.82 

2.35 

C_WORK 

4.80 

0.76 

5.00 

1.87 

2.60 

1.52 

4.13 

1.76 

R_WORK 

5.50 

1.46 

6.00 

1.22 

4.00 

1.87 

5.17 

1.68 

E_WORK 

6.88 

0.85 

7.25 

0.50 

5.00 

1.63 

6.38 

1.43 

SCENARIO  1 
T/0  Interval  Data 


Control  Position 

Group  Total 

R 

D 

Mean 

Std.  Dev. 

Mean 

Std.  Dev. 

Mean 

Std.  Dev, 

SECTOR 

R63 

C_WORK 

2.10 

1.48 

2.20 

1.47 

2.15 

1.46 

R_WORK 

2.70 

1.49 

2.70 

1.75 

2.70 

1.60 

E.WORK 

4.25 

2.45 

4.58 

2.02 

4.42 

2,21 

R60 

C_WORK 

1.75 

0.85 

1.75 

0.85 

1.75 

0.84 

R_WORK 

1.65 

0.81 

1.60 

0.75 

1.63 

0.77 

E_WORK 

4.00 

2.37 

3.58 

2.27 

3.79 

2.28 

Group  Total 

C_WORK 

1.93 

1.21 

1,98 

1.21 

1.95 

1.20 

R_WORK 

2.17 

1.30 

2.15 

1.44 

2.16 

1.36 

E_WORK 

4.13 

2.36 

4.08 

2.17 

4.10 

2.24 

SCENARIO  2 
T/0  Interval  Data 


Control  Position 

Group  Total 

R 

D 

Mean 

Std.  Dev. 

Mean 

Std.  Dev. 

Mean 

Std.  Dev. 

SECTOR 

R60 

C.WORK 

2.80 

1.79 

2.50 

1.57 

2.65 

1.67 

R_WORK 

2,45 

1.50 

2.45 

1.39 

2.45 

1.43 

E_WORK 

3.63 

2.39 

4.00 

1.77 

3.81 

2.04 

R62/63 

C_WORK 

2.65 

1.90 

3.00 

1.92 

2.83 

1.89 

R_WORK 

2.65 

1.50 

3.05 

1.88 

2.85 

1.69 

E_WORK 

3.00 

1.69 

3.38 

1.60 

3.19 

1.60 

Group  Total 

C_WORK 

2.73 

1.83 

2.75 

1.75 

2.74 

1.78 

R_WORK 

2.55 

1.48 

2.75 

1.66 

2.65 

1.57 

E_WORK 

3.31 

2.02 

3.69 

1.66 

3.50 

1.83 
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SCENARIO  3 
T/0  Interval  Data 


Control  Position 

_ _ - . . . . 

Group  Total 

R 

D 

Mean 

Std.  Dev. 

Mean 

Std,  Dev. 

Mean 

Std.  Dev. 

SECTOR 

R60/62/63 

C_WORK 

2.35 

1.41 

2.55 

1.66 

2.45 

1.52 

R_WORK 

2.50 

1.56 

2.55 

1.70 

2.53 

1.61 

R1 

C_WORK 

2.25 

1.41 

2,05 

1.39 

2.15 

1.39 

R_WORK 

2.53 

1.52 

2.90 

2.02 

2.71 

1.78 

Group  Total 

C_WORK 

2.30 

1.39 

2.30 

1.54 

2.30 

1.46 

R_WORK 

2,51 

1.52 

2.73 

1.85 

2.62 

1.69 

SCENARIO  4 
T/0  Interval  Data 


Control  Position 

Group  Total 

R 

D 

A 

Mean 

Std.  Dev. 

Mean 

Std.  Dev. 

Mean 

Std.  Dev. 

Mean 

Std.  Dev. 

C_WORK 

3.60 

1.91 

3.90 

2.25 

1.90 

1.55 

3.13 

2.09 

R_WORK 

3.95 

2.33 

4.25 

1.89 

1.95 

1.32 

3.38 

2.12 

E_WORK 

5.34 

2.77 

5.31 

2.98 

3.94 

2.49 

4.86 

2.77 

SCENARIO  I 
SAR  Data 


Position 

Group  Total 

R 

D 

Mean 

Std.  Dev. 

Mean 

Std.  Dev. 

Mean 

Std.  Dev. 

SECTOR 

R63 

C_QN 

127.20 

23.09 

0.00 

0.00 

63.60 

68.79 

R_QN 

118.40 

22.79 

0,00 

0.00 

59.20 

64.22 

E_0N 

74.00 

64.21 

0.00 

0.00 

37.00 

57.38 

C_QT 

15.20 

5.02 

0.00 

0.00 

7.60 

8.68 

R_OT 

14.20 

3.11 

0,00 

0.00 

7.10 

7.77 

E„QT 

5.00 

4.36 

0.00 

0.00 

2.50 

3.89 

C_QU 

6.80 

2.77 

5.60 

3.85 

6,20 

3.22 

R_QU 

6.20 

2.68 

5.80 

3.27 

6.00 

2.83 

E_QU 

5.00 

4.58 

1.33 

1.53 

3.17 

3.66 

C_QX 

4.60 

1.67 

0.20 

0.45 

2.40 

2.59 

R_0X 

4.00 

1.22 

0.60 

0.89 

2.30 

2.06 

E__QX 

1.67 

1.53 

0.00 

0.00 

0.83 

1,33 

C_QZ 

5.40 

0.89 

10.60 

13.43 

8.00 

9,38 

! 

R_QZ 

7.20 

2.49 

7.80 

9.15 

7.50 

6.33 

E_OZ 

7.67 

6.81 

4.33 

5.86 

6.00 

5.97 

R60 

C_ON 

123.00 

28.52 

0.00 

0.00 

61.50 

67.56 

R_QN 

98.40 

5.55 

0,00 

0.00 

49.20 

51.99 

E_QN  ; 

75.67 

71.46 

0.00 

0.00 

37.83 

61.32 

C_OT  1 

11.60 

4.56 

0.20 

0.45 

5.90 

6.74 

R_QT 

8.20 

0.84 

0.20 

0.45 

4.20 

4.26 

E_0T 

2.67 

2.52 

0.00 

0.00 

1.33 

2.16 

D-16 


lililH 

9.1  \ 

ESZSHIB 

6.54 

10.50 

7.81 

LiitH 

12.00 

Q13HB 

4.93 

9.70 

4.90 

7.94 

EE&^H 

6.56 

8.00 

6.60 

MtVf^ 

1.92 

EQ^HBi 

0.45 

1.50 

1.90 

0.45 

0.80 

.92 

3.33 

2.89 

El^HflB 

0.00 

1.67 

2.58 

2.35 

msmt 

7.44 

10.20 

5.69 

1.30 

EE^IHI 

8.26 

12.00 

5.58 

■■■■ 

6.00 

7.94 

5.83 

6.31 

0.00 

0.00 

62.55 

66.36 

R_QN 

0.00 

0.00 

54.20 

57.10 

l»!SH 

0.00 

0.00 

37.42 

56.62 

13.40 

4.90 

0.10 

0.32 

6.75 

7.61 

11.20 

3.82 

0.32 

5.65 

6.28 

3.83 

3.43 

eeshh 

0.00 

1.92 

3.06 

7.02 

5.66 

8.35 

6.22 

EBE^^H 

4.46 

4.03 

7.85 

4.33 

4.17 

5.27 

5.58 

5.68 

liltM 

1.95 

0.42 

1.95 

2.26 

■«»V 

IKHBB 

0.70 

1.55 

1.73 

■J>W4M 

2.50 

2.26 

0.00 

0.00 

1.25 

2.01 

Bsn 

2.16 

11.50 

10.28 

9.10 

7.64 

■aiH 

10.00 

8.54 

9.75 

6.25 

6.67 

5.85 

5.17 

6.31 

5.92 

5.85 

SCENARIO  2 
SAR  Data 


Position 

Group  Total 

R 

D 

Mean 

Std.  Dev. 

Mean 

IRfnSffnH 

Mean 

Std.  Dev. 

SECTOR 

R60 

C_ON 

99.60 

57.57 

0.00 

0.00 

49.80 

65.03 

0.00 

65.67 

72.42 

E_0N 

70.00 

98.99 

C_QT 

7.60 

3.80 

5.12 

R_OT 

8.67 

4.33 

4.93 

2.00 

2.83 

BPHi 

7.07 

3.60 

4.39 

7.30 

6.78 

R_QU 

14.33 

5.13 

2.67 

2.31 

8.50 

7.31 

E_OU 

6.00 

6.00 

6.00 

0.00 

3.00 

eeshh 

0.00 

1.50 

2.07 

2.33 

EE&HH 

0.00 

1.17 

1.33 

EESHH 

2.00 

2.83 

c_oz 

6.40 

4.39 

i^BBI 

15.65 

8.00 

10.96 

R_0Z 

10.67 

1.15 

4.67 

3.79 

7.67 

4.13 

E_OZ 

9.00 

1.00 

5.00 

5.66 

R62/63 

C_QN 

92.40 

0.00 

0.00 

46.20 

60.25 

0.00 

42.50 

60.10 

15.60 

11.33 

EEEIIflHBI 

11.16 

9.85 

EESl^fli 

8.67 

E_QT 

11.00 

0.00 

5.50 

7.78 

c_ou 

10.00 

6.89 

0.40 

0.55 

5.20 

6.84 

R_OU 

7.00 

8.19 

0.33 

0.58 

3.67 

6.35 
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IsIdH 

11.00 

5.50 

7.78 

EBB 

5.00 

2.92 

0.55 

2.70 

3.13 

4.67 

4.16 

2.33 

3.67 

IsBB 

9.00 

4.50 

6.36 

1.95 

0.55 

1.60 

1.71 

1.33 

3.46 

■■■■ 

4.00 

2.83 

MrtiM 

EESHI^I 

EESHH 

ISB 

51.61 

EESlHi 

E_QN 

ee&hhi 

EESHH 

C_QT 

9.22 

ebb 

R_QT 

9.83 

6.49 

0.00 

0.00 

4.92 

6.75 

E_QT 

7.50 

4.95 

0.00 

0.00 

3.75 

5.19 

C_QU 

10.50 

6.60 

2.00 

3.40 

6.25 

6.72 

R_QU 

10.67 

7.31 

1.50 

1.97 

6.08 

7.00 

E_QU 

8.50 

3.54 

3.00 

4.24 

5.75 

4.50 

C_QX 

4.00 

2.58 

0.20 

0.42 

2.10 

2.65 

R_QX 

3.50 

2.95 

0.00 

0.00 

1.75 

2.70 

E_QX 

6.50 

3.54 

0.00 

0.00 

3.25 

4.27 

C„QZ 

4.50 

3.78 

5.10 

11.46 

4.80 

8.31 

R_QZ 

7.67 

4.23 

3.00 

3.35 

5.33 

4.38 

E_QZ 

7.50 

2.12 

1.50 

0.71 

4.50 

3.70 

SCENARIO  3 
SAR  Data 


Position 

Group  Total 

R 

D 

Mean 

Std.  Dev. 

Mean 

Std.  Dev. 

Mean 

Std.  Dev. 

SECTOR 

R60/62/63 

C_QN 

115.50 

20.66 

0.00 

0.00 

57.75 

63.20 

R_0N 

71.33 

61.98 

0.00 

0.00 

35.67 

55.34 

C_QT 

13.50 

4.20 

0.00 

0.00 

6.75 

7.72 

R_QT 

7.00 

6.08 

0.00 

0.00 

3.50 

5.43 

C_QU 

15.00 

3.16 

4.50 

4.36 

9.75 

6.63 

R_QU 

10.67 

11.02 

8.00 

3.46 

9.33 

7.45 

C_QX 

3.25 

.50 

0.00 

0.00 

1.63 

1.77 

R_QX 

1.67 

1.53 

0.33 

0.58 

1.00 

1.26 

C_QZ 

6.00 

2.71 

1.75 

1.50 

3.88 

3.04 

R_QZ 

6.33 

5.51 

2.33 

2.08 

4.33 

4.32 

R1 

C_QN 

120.25 

9.18 

0.00 

0.00 

60.13 

64.56 

R„QN 

115.33 

5.86 

0.00 

0.00 

57.67 

63.28 

C_QT 

5.75 

2.06 

1.25 

1.89 

3.50 

3.02 

R_QT 

6.33 

3.51 

1.33 

1.53 

3.83 

3.66 

C_QU 

8.00 

3.37 

4.25 

5.32 

6.13 

4.58 

R  QU 

7.00 

2.65 

5.00 

7.81 

6.00 

5.33 

C_QX 

1.25 

:  1.50 

0.25 

0.75 

1.16 

R_0X 

1.00 

^  1.00 

0.00 

EESHH 

0.50 

C_QZ 

10.00 

2.83 

19.50 

11.47 

9.25 

R_0Z 

10.33 

3.06 

20.67 

10.11 

Group  Total 

C_QN 

117.88 

15.02 

0.00 

0.00 

58.94 

61.73 

R_QN 

93.33 

46.16 

0.00 

0.00 

46.67 

57.83 

C_QT 

9.63 

5.15 

0.63 

1.41 

5.13 

5.91 

D-18 
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APPENDIX  E 


Possible  Conflicts  Noted  by  the  Technical  Observers 


Table  El  lists  all  the  possible  conflict  situations  recorded  on  the  T/0  form  during  the  simulation. 
Situations  were  classified  as  operational  errors  or  Deviations  according  to  the  71 10.65  Air 
Traffic  Controller  Handbook.  “Situations”  were  denoted  as  “RVSM”  if  there  was  a  separation 
violation  that  was  not  a  violation  under  current  standards  but  would  have  been  under  reduced 
vertical  standards.  The  word  “None”  appears  if  a  situation  was  recorded  but  did  not  fall  into  any 
of  the  previously  mentioned  categories.  The  day,  scenario,  sector,  and  interval  for  each  situation 
are  also  listed.  The  last  column  of  the  table  lists  the  control  team  responsible  for  the  possible 
conflict  and  the  total  number  of  errors  that  particular  team  made  for  the  day.  For  example,  the 
Deviation  listed  in  row  one  was  made  by  control  team  K.  On  the  25“’,  team  K  had  a  total  of  6 
possible  conflict  situations. 


Table  El .  Possible  Conflicts  Noted  by  the  T/Os 


Interval 

Situation 

Type 

Explanation 

Team- 

#Errors 

25 

1C 

R60 

2 

Deviation 

FL  310  coordinated  on  CUB484  but  the  strip  was  not 
cocked  out. 

K-6 

25 

1C 

R60 

3 

Deviation 

FL  330  coordinated  on  IBE620  but  the  strip  was  not 
angulated. 

K-6 

26 

1C 

R63 

2 

Error 

BAW265  &  LTU422  -  BAW265  checked  in  at  FL  330 
with  non  radar  traffic  (LTU442)  4  minutes  in-trail  at  FL 
350  that  was  not  identified  or  talking  to  ZMA. 

BAW265  climbed  to  FL  390  through  the  protected 
"push"  ofLTU442.  (Non-radar  error) 

N-1 

25 

1C 

R63 

4 

None 

Two  aircraft  were  head-on  at  FL  310.  The  controller 
took  the  proper  action  to  resolve  the  conflict  and  also 
corrected  the  situation  in  a  safe  amount  of  time. 

L-l 

20 

IE 

Radar 

R60 

Error 

AAL638  &  AAL509  -  When  radar  outage  occurred, 
AAL638  was  at  FL  330  and  AAL509  was  at  FL  330. 
Both  aircraft  were  proceeding  direct  ROBLE 
approximately  2  minutes  in  trail.  Non-radar  separation 
was  never  established. 

G-2 

20 

IE 

Radar 

R60 

3 

Error 

CDN357  &  AAL267  -  CDN357  proceeded  direct  to 
ALBEE.  AAL267  wasatFL330on  A315.  The 
aircraft  were  radar  separated  at  the  time  of  the  radar 
outage  non-radar  longitudinal  separation  of  ten  minutes 
in-trail  was  not  established. 

G-2 

16 

IE 

Radar 

R60 

4 

Error 

CUB 944  &  AAL638  -  CUB 944  was  routed  direct 
POTAR.  AAL638  was  routed  direct  ROBLE  at 
WAFDOF  (FL  350  climbed  from  FL  330).  Controller 
did  not  request  DME  from  any  VOR  to  see  if  aircraft 
were  laterally  separated. 

A-2 

25 

R60 

2 

Deviation 

ROY  173  was  WAFDOF  and  the  strip  was  not  marked. 

K-6 

E-1 


Interval 


Situation 

Type 


Explanation 


AAL638  &  AAL507  -  Controller  took  hand-off  for 
AAL638,  which  was  converging  with  AAL507  that  was 
deviating  because  of  weather.  The  controller  took  too 
long  to  resolve  possible  conflict. 


AAL509  &  AAL638  -  Potential  conflict  with  AAL509 
and  AAL638.  R  side  kept  calling  AAL509  AAL508. 


AAL507  &  CON357  -  Possible  conflict.  The 
controller  never  saw  it. 


AAL507  &  AAL638  -  AAL507  turned  to  GRATX  at 
ZQA.  It  crossed  in  front  of  AAL638  at  the  same 

altitude  without  the  required  separation. _ 

BAW297  &  IBE621  -  Aircraft  were  converging. 
IBE621  was  at  FL  330  and  BAW297  was  climbing  to 
FL  350.  The  target  touched  the  5-mile  buffer  of  the 
other  aircraft.  Closest  proximity  of  4.5  miles  and  1,500 
ft  not  in  WATRS  area. 


Team- 

#EiTors 


19 

3C 

R60/62/63 

N/G 

Deviation 

No  track  was  started  for  DOA964.  ACU522  had  2 
altitudes  written  on  the  strip.  Numerous  aircraft  did  not 
have  oceanic  "over"  circles. 

E-3 

19 

3C 

R60/62/63 

N/G 

Deviation 

SUV605  -  serve  USA  six  hundred  five. 

E-3 

16 

3C 

R60/62/63 

2 

Error 

AAL57  &  UAL053  -  Both  aircraft  were  over  GRATX 
at  FL  390  with  only  5  minutes  separation. 

A-2 

18 

3R 

R1 

2 

Error 

CMM509  &  AFL389  -  CMM509  was  southbound  on 
AR3  at  FL  350  (NUCAR  time  00:21).  AFL389  was 
inbound  on  A699  at  FL  350  (NUCAR  time  00:19). 

C-2 

18 

3R 

R1 

2 

Error 

CMM509  &  DLH464  -  CMM509  was  southbound  on 

AR3  climbing  to  FL  370  (NUCAR  time  00:21). 

DLH464  was  inbound  on  A699  at  FL  370  (NUCAR 

TIME  00:24). _ 

'  CMM509  &  AFL389  -  AFL389  was  at  FL  350  and 
CMM509  was  at  FL  350.  AFL389  should  have  been  at 
FL  330  as  per  coordination.  Aircraft  were  less  than  10 

minutes  apart. _ 

Controller  issued  EGF938  a  clearance  without  stating  an 

altitude. _ _ 

LTU420  &  ROY  173  -  LTU420  was  approved  at 
WATRS  boundary  at  FL  310.  ROY  173  was  coming 
from  SJU  at  FL  3 10.  Both  aircraft  crossed  within  4 

minutes  in  non-radar  airspace  at  01 :36. _ 

CKS869  &  LHN861  -  CKS869  was  estimated  over 
SEKAR  at  02:28  at  FL  330.  LHN861  was  estimated 
over  SEKAR  at  02:35  at  FL  330.  There  was  not  enough 

time.  It  was  done  by  the  A-side. _ 

DART22  &  M3389  -  DART22  was  at  FL  370  over 
GEJAY  and  M3398  was  over  SLAPP  at  same  time. 
SPP995  was  southwest  bound  on  B  891  at  FL  350 
descending  to  FL  290  with  non-radar  traffic  8  minutes 
in-trail  at  FL  310.  (Non-radar  Error) 


E-2 


Interval  Situation 
Type 


25 

4C 

R62/63 

19 

4E 

Radar 

R62/63 

23 

4E 

Radar 

R62/63 

23 

4E 

Radar 

R62/63 

18 

4E 

Radar 

R62/63 

18 

4E 

Radar 

R62/63 

4E 

R62/63 

4E 

Radar 

R62/63 

23 

4E 

Radar 

R62/63 

23 

4E 

Radar 

R62/63 

23 

4E 

Radar 

R62/63 

Deviation 


4E 

Radar 

R62/63 

4E 

Weather 

R62/63 

4E 

Weather 

R62/63 

Explanation 


DOA474  &  AJT840  -  Controller  climbed  DOA474 
through  AJT84Q  and  only  had  about  7  miles. 


Too  many  operational  errors  to  write  down  due  to  radar 
OTS _ 


CMM314  &  AAL141  -  CMM314  over  SLAPP  at  22:48 
and  FL  400.  AAL141  over  FORD  at  22:50  and  also  at 
FL  400.  Fifteen  degrees  divergence  was  coordinated 
but  R  side  issued  direct  SLAPP  to  CMM314,  which 

nullified  the  15-degree  divergence. _ 

USA041  wrong  altitude  for  direction  of  flight  not 

coordinated  with  New  York. _ 

RVSM  //  Controller  assigned  CMM  314  to  FL  380  15 

miles  north  of  BOTES.  Not  RVSM  airspace. _ 

RVSM  //  Controller  assigned  AAL658  to  FL  320  30 
miles  northeast  of  BOTES.  (Not  RVSM  airspace) 


Team- 

#Errors 


VIA828  &  AAL720  -  VIA828  was  heading  northwest  to 
GT  at  FL  350  and  merged  with  AAL720  heading  north 
to  GT  at  FL  350. 


AAL658  &  CMM314  -  When  radar  was  lost  both 
aircraft  were  at  FL  350.  There  was  no  longitudinal 
separation. _ 


AAL401  &  AAL679  -  Both  aircraft  were  at  FL  370 
when  radar  was  lost.  There  was  no  longitudinal 
separation  _ 


ARU758  &  VIA822  -  ARU758  was  at  FL  280  on  A3 15 
estimating  ZIN  at  22:30.  VIA822  was  at  FL  280 
estimating  ZIN  at  22:22.  (Need  10  minutes) 


CWC173  &  VAL176  -  CWC173  was  at  FL  350  on 
A315  estimating  ZIN  at  22:30.  VAL176  was  at  FL  350 

estimating  ZIN  at  22:22.  (Need  10  minutes) _ 

LHN892  wrong  altitude  for  direction  of  flight  not 
coordinated  with  New  York. 


Controller  had  incorrect  coordination  strip.  Needed 
SLAPP  but  had  BOTES.  Controller  estimated  the 
correct  time  at  SLAPP  to  be  0109Z  put  that  time  on  the 
BOTES  strip,  changed  the  name  BOTES  to  SLAPP  and 
coordinated  that  time.  When  he  got  the  correct  SLAPP 
strip  the  correct  estimated  time  was  005 8Z.  The 

revised,  correct  time  was  not  re-coordinated. _ 

'  DOA472  &  SPP896  -  DOA472  descended  to  FL  270  to 
land  at  MDPP.  SPP896  climbed  to  FL  280  from  a  non¬ 
radar  departure  at  MDPP.  SPP896  should  have  been 
stopped  at  a  lower  altitude. 


E-3 


Interval 


Situation 

Type 


Explanation 


Team- 

#Errors 


Error  AAL141  &  CTC410  -  Radar  was  lost  with  CTC410 
descending  to  FL  370  and  AAL141  at  FL  390.  (non¬ 
radar  error). 


RVSM  RVSM  //  Use  of  RVSM  in  non-WATRS  area. 

Error  USA  167  &  M3390  -  USA  167  was  climbing  out  of  FL 

350  to  FL  370  over  SLAPP  at  20:43.  M3390  was  at  FL 
370  over  SLAPP  at  20:51.  (Less  than  10  minutes 
separation) 


23 

4R 

R62/63 

2 

Deviation 

ROY  173  was  northbound  at  FL  310  and  not 
coordinated  as  wrong  altitude  for  direction  of  flight  with 
New  York. 

I-IO 

24 

4R 

R62/63 

2 

Deviation 

AAL719  was  in  the  wrong  bay  and  was  not  coordinated 
with  Santo  Domingo.  Error  was  corrected  at  22:18 

J-8 

18 

4R 

R62/63 

3 

Error 

APA283  &  AAL657  -  APA283  was  over  SLAPP  at 
19:27  and  AAL657  was  over  GEJAY  at  19:32.  Both 
were  at  the  same  altitude  without  non-radar  separation. 

D-6 

18 

4R 

R62/63 

i 

3 

RVSM 

RVSM  //  AAL728  cleared  to  FL  360  40  miles  south  of 
GTK.  Not  in  RVSM  Area.  AC  A  reached  FL  360  at 
19:45  15  miles  north  of  GTK.  Later  descended  to  FL 
340  30  miles  north  of  GTK. 

D-6 

18 

4R 

R62/63 

3 

Deviation 

AAL216  entered  with  no  handoff. 

D-6 

24 

4R 

R62/63 

3 

Error 

ROY  173  &  APA283  -  New  York  requested  FL  310  for 

J-8 

18  4R 


23  4R 


ROY173.  D-side  showed  FL  320  at  22:25.  CFG158 
was  over  SEKAR  at  22:29  descending  to  FL  110. 
APA283  over  SEKAR  at  22:40  descending  to  1 1,000  ft. 

Need  15  minutes. _ 

RVSM  //  DOA241  entered  non-RVSM  area  at  FL  380. 


DOA474  &  N404PA  -  DOA474  and  N404PA  were 
both  at  FL  330  very  close  to  WATRS  intersection.  Non 
radar  separation  was  not  ensured. 


